Re: summary of recent important developments (ultra-compressed, random)

Interesting post.

The list of fragments taken as a whole implies a medium-term big-picture fu=
ture that conforms poorly to broad socio-political expectations. I expect p=
eople will react poorly to that environment while simultaneously adapting t=
o it and taking advantage of the potential benefits. =


Some additional comments and fragments of my own below.

On Mar 17, 2014, at 12:41 AM, Kragen Javier Sitaker <[email protected]> =
wrote:
> - Rust and Golang seem to be pretty hot, but Clojure is hotter.


Related:

- Performance per watt is starting to drive programming language usage and =
software design patterns, similar to how it already drives silicon design. =
Some traditional software wisdom and idioms show poorly in this context. Ma=
ximizing compute per watt will transform software like it did hardware.

- Garbage collectors are being replaced by local custom allocators in serve=
r infrastructure software, even in applications where GCs have traditionall=
y been used, due largely to significant differences in efficiency. Language=
s like Rust have an advantage here. =



> - Tick-to-trade times in HFT have fallen into the single-digit microsecond
>  range, and HFT people are doing things like userspace NIC drivers in ord=
er to
>  stay in the game.  That playing field is due for some serious consolidat=
ion
>  this year.


- Intel embedding NICs in CPUs starting this year will move most networking=
 into userspace. Kernel networking is one of the biggest efficiency drains =
in server software today and userspace NIC drivers are not unknown in ordin=
ary high-efficiency server software. With a =93standard=94 NIC-in-the-CPU, =
this approach suddenly becomes much simpler.


> - The transition of marketed energy to photovoltaic started happening in
>  earnest in 2013 and will be complete in the mid-2020s, which could resul=
t in
>  a bigger (log-scale) increase in marketed energy consumption than the
>  Industrial Revolution, since the solar energy resource is four orders of
>  magnitude bigger than current world marketed energy consumption.


This photovoltaic energy will be necessary to drive the compute applied to =
the (also log-scale) increase in data being generated by autonomous real-ti=
me sensor networks over the same period. These markets are already severely=
 power-limited at every scale and we=92ve barely even started. Massively pa=
rallel ARM clusters are already being used in such applications but don=92t=
 show that well for non-naive code; it is driving a lot of compute-per-watt=
 research.

Related:

- The machine-scale Internet is about to eclipse the human-scale Internet. =
Software infrastructure built for the human-scale Internet and analysis (i.=
e. all of it) requires a complete rewrite to support the machine-scale phas=
e of the market. =


- We are within a few years of being able to build detailed predictive beha=
vioral models of every single individual in the industrialized world and ap=
plying them in real-time, including aggregate behaviors of populations. Non=
e of our social, cultural, or political institutions are prepared for this =
nor its implications.


> - The free software movement has died.
> - Due to Bitcoin and Snowden, computer security became more than a PR iss=
ue,
>  providing some hope that it will become possible to provide it within the
>  economic system, but not much.  The recent GnuTLS bug, which survived al=
most
>  ten years of scrutiny by the proverbial "many eyes", suggests that Benkl=
er's
>  commons-based peer production is not currently capable of providing it
>  either.  That leaves firms, which is to say, planned economies.


In my opinion, the free software movement has run into the confluence of a =
few trends that are effectively marginalizing its practical relevance in so=
me areas. In no particular order:

- Cloud computing and the model of wiring together distributed abstractions=
 reduced one inefficiency (idle or inefficiently utilized resources) but re=
placed it with another (inefficient computation). Cloud computing infrastru=
cture companies have an incentive to improve efficiency by optimizing away =
abstractions and flexibility via modifications that break the modularity of=
 open source. This creates an infrastructure cost advantage at low cost for=
 the modified platform.

- To the point of GnuTLS, one of the explicit reasons to deploy custom clos=
ed source, particularly in cloud environments, is to significantly reduce a=
nd control the practical attack surface. A lot of open source software was =
not designed with security in mind, sometimes at an architectural level. I=
=92ve seen a movement to replace many third-party open source dependencies =
with carefully designed alternatives specifically engineered for the scope =
of their application. Security through obscurity, particularly when combine=
d with careful code verification, can work well.

- Intellectual property related to computer science R&D is largely unenforc=
eable for all practical purposes but modern models of distribution =97 appl=
iances and the cloud =97 make trade secrets an effective way to gain a sust=
ainable competitive advantage. One of the practical disadvantages of open s=
ource is that you see little pure technical innovation originating from tha=
t movement but it is difficult to copy innovations that are never published=
 or inspectable.


Combined with the above point that the machine-scale Internet requires a co=
mplete rewrite of the infrastructure software, it is easy to arrive at the =
conclusion that within five years most of the infrastructure software that =
runs the Internet will be closed source and essentially invisible. =



> - OKCupid added "relationship type: non-monogamous" and Facebook added do=
zens
>  of "custom" genders this month.  Geek feminism, if that's what that is, =
is
>  getting things done.



This makes me wonder if it reflects a deeper change in how individuals in s=
ociety relate to each other. Traditional, socially acceptable relationships=
 are a lot like traditional, socially acceptable careers; you were born int=
o a limited number of options, chose young, and stuck with it for the rest =
of your life. The assumptions upon which this is based have been questionab=
le as universals for some time, and quite noticeably so in the tech communi=
ty.

Everything is moving toward distributed, non-hierarchical, ad hoc organizat=
ions and identities (see also: bitcoin) that exist for as long as it makes =
sense to the involved parties because it is economically efficient. It has =
been interesting to see social and cultural institutions slowly evolve in t=
hat direction because they were never designed to work in that world. =




_______________________________________________
FoRK mailing list
http://xent.com/mailman/listinfo/fork