All posts by Rufo Guerreschi

Comune di Roma leader mondiale nella democrazia elettronica digitale?!

UPDATE May 20th 2016: una versione estesa ed aggiornata di questo post la trovi qui.

Con le elezioni comunali di Roma del 2016 è per la prima volta possibile che il Movimento Cinque Stelle possa controllare ed amministrare un grande comune d’Italia, il più grande. Sarà un’enorme occasione e responsabilità per dimostrare di poter mettere in pratica i principi di partecipazione, trasparenza e lotta contro corruzione e privilegi tanto affermati fino ad oggi dall’opposizione.

Sarà altresì occasione di implementare con risorse adeguate e sul larga scala, i principi della democrazia diretta e partecipativa fortemente affermati dal movimento, attraverso innovative politiche, tecnologie e processi.

Sarà interessante vedere quali sistemi il Comune di Roma deciderà di adottare, ovviamente attraverso bandi di gara, e in che misura soluzioni attualmente disponibili sul mercato possano fornire livelli di inclusività, di sicurezza e privacy che richiede la Costituzione Italiana, la Carta Europea dei Diritti dell’uomo, e probabilmente i cittadini di Roma, e una buona parte del 97% dei votanti romani del Movimento che hanno scelto di non utilizzare gli strumenti di partecipazione online gratuiti predisposti dal Movimento.

L’amministrazione entrante potrebbe prendere atto della mancanza di tali strumenti e quindi attivare da una parte bandi di gara a breve termine per una fornitura di strumenti per un utilizzo in bassa-scala e/o sperimentale, ed al contempo bandi di gara a lungo termine – in linea e coordinamento con programmi EU pre-commercial procurement, che includano un sostanziale livello di ricerca applicata e sostanziale innovazione di sistemi esistenti per soddisfare i requisiti di cui sopra.

“Head of NSA’s Elite Hacking Unit: How We Hack” by ABC News

ABS News come out with this article:

http://abcnews.go.com/International/head-nsas-elite-hacking-unit-hack/story?id=36573676

In assessing the veracity and completeness of what the head of NSA Tao says here we should consider it’s important part of its agency mission that hundreds of thousands of potential mid-to-high targets, legit or not, overestimate the manual (as opposed to semi automated) resources and efforts they need to devote per person to continuously compromise them. See NSA FoxAcid and Turbine programs.

So these targets will think NSA “endpoint target list” is small and does not include them, and therefore are fine with end-2-end encryption, and merely moderate or high assurance endpoints, like Tor/Tail, Signal on iOS, or an high end cryptophone.

Il documentario su Garry Davis, l’assemblea costituente globale e l’intelligenza artificiale

Perchè la visione di Garry Davis di un’assemblea costituente globale è più attuale oggi di allora, oltre 70 anni dopo, dinanzi a minacce per l’umanità ancor maggiori del rischio nucleare.  

E’ iniziato un tour mondiale per la visione della versione pre-finale del documentario “My Country is the World” (“la mia patria è il mondo”) sull’incredibile avventura del primo cittadino del mondo, Garry Davis.

Il documentario, di 79 minuti, incomincia con una testimonianza dall’attore Martin Sheen, che partecipa alla promozione di un progetto collegato di film che è in via di realizzazione. Ecco un trailer di 7 minuti su youtube.

Nel novembre 1948, con una storica intuizione, Garry Davis, attore di Broadway ed ex-pilota USA durante la seconda guerra mondiale, rinunciò volontariamente alla propria nazionalità nel 1945 a Parigi, per dichiararsi cittadino del mondo. Fu il primo cittadino americano della storia a farlo, e prese residenza nella terra franca della prima sede dell’ONU a Parigi in una tenda, chiedendo l’effettiva costituzione di una democrazia globale.

Personalità come Jean Paul Sartre, Eleanor Roosvelt, Camus ed Einstein lo indicarono come simbolo di quello che l’ONU sarebbe dovuto e potuto diventare in quei mesi chiave. Seguendo l’esortazione di Eleanor Roosvelt di “avviare senza indugio un governo mondiale internazionale1, Garry divenne protagonista di un grande movimento sociale e politico che – con manifestazioni di decine di migliaia di persone e azioni spettacolari – chiedeva la convocazione di una vera e propria Assemblea costituente mondiale.

Da allora, creò un servizio di emissione di passaporti mondiale, ed un movimento di cittadini del mondo, che distribuì oltre 950.000 passaporti a cittadini del mondo.

Viaggiò, promosse e scrisse fino all’età di 91 anni, fedele alla sua visione, traversando dozzine di paesi con il passaporto mondiale e attraversando le frontiere diverse decinedi volte. Dopo alcuni decenni il documento è stato riconosciuto formalmente da dozzine di paesi, come documento d’identità secondario, ed in pochi casi come primario.

Tra gli articoli sulla stampa internazionale in occasione della sua morte avvenuta nel 2013 vi sono, quello del New York Times, del Telegraph, oltre ad un breve pezzo dell’Internazionale.

Einstein ebbe a dire: “Mark my words, this boy, Garry Davis, has grasped the only problem which deserves the devotion of contemporary man… the survival of the species. It is a question of knowing whether mankind – the universe of man – will disappear by its own hand, or whether it will continue to exist.”

Dalla rivoluzione francese e americana, le democrazie nazionali sono sempre state create attraverso assemblee costituenti, con rivoluzioni o sommovimenti più o meno violenti. E’ tmepo di riconoscere senza mezzi termini che l’approccio incrementale adottato per la creazione di istituzioni democratiche a livello internazionale è stato completamente fallimentare sia a livello europeo che a livello delle Nazioni Unite.

Dopo tanti decenni, gli stati economicamente più forti (nel caso dell’UE) e i vincitori dell’ultima guerra mondiale (nel caso dell’ONU) si sono consolidati in forme di egemonia oligopolistica con piccoli passi avanti nei processi di integrazione per dare l’impressione di un progresso che, dopo 70 anni, possiamo dire con certezza di essere completamente immaginario.

Oggi come allora,  la via verso la democrazia mondiale ed europea passa per un pieno processo costituente transnazionale. Non ci sarebbe nessun bisogno di forzare gli Stati che possono decidere di non aderire attraverso un referendum popolare, o con decisione parlamentare. Basterebbe che gli Stati che vogliono andare avanti creino una prima assemblea costituente transnazionale, aperta a future adesioni con le stesse condizioni. Il tutto potrebbe anche avvenire all’interno delle Nazioni Unite attraverso un UN caucus dedicato, come delineato in una proposta2 approvata dall’assemblea generale del World Federalist Movement nel 2007.

Ma tutto ciò, come intuì Garry Davis, non potrà mai avere successo, senza un supporto di un vasto movimento sociale e di singoli cittadini globali – oltrechè di città – che solo può fornire i necessari strumenti di controllo e di rappresentatività dei processi costituenti. Ancora Einstein disse “A supra-national institution must have enough powers and independence if it shall be able to solve the problems of international security. Neither can one nor has one the right to leave the taking of such a decisive step entirely to the initiative of the governments.

C’è bisogno di movimento che non chieda riforme graduali ma, con chiarezza e determinazione, l’unico primo passo passo verso la democrazia mondiale: la democrazia mondiale stessa. Davis ha sempre considerato che il più grande ostacolo alla creazione di una democrazia globale è stata la proprio la focalizzazione di Stati e e individui nel migliorare in modo incrementale processi costituenti gravemente inadeguati avviati con l’UE e l’ONU.

Molto più che semplicemente disperdere energie, la legittimazione di processi tali costituenti corrotti e castrati sin dalla nascita – spesso per precisi interessi geo-strategici di breve o lungo periodo – ha impedito più di ogni altra cosa la costituzione di veri processi costituenti globali.

Ma negli ultimi anni, con l’incredibile accelerazione dello sviluppo delle tecnologico, l’umanità non può più permettersi di essere guidata da poche centinaia di super ricchi, finanzieri e imprenditori informatici di enorme successo, e dagli interessi dei governi dei paesi più forti, spesso enormemente influenzati da potentati economici, mass-mediatici, e militari-industriali.

La posta oggi è infatti è ancora più alta del rischio nucleare che guidò e sospinse la creazione dell’ONU dopo la seconda guerra mondiale.

Ai rischi nucleari,sotto molti aspetti irrisolti e accresciuti, si sono aggiunti crescenti e sinergici rischi di catastrofi umanitarie – come cambiamenti climatici, guerre ibride, nuove pandemie, nonchè prospettive di “consolidamento di forme durevoli di governanza globale inumana1 . 

Ma il rischio supremo, che si spera potrà dare una spinta sufficiente per la creazione di una democrazia mondiale, è il rischio per la sopravvivenza stessa dell’umanità, nell’orizzonte dei prossimi 10-70 anni, dovuto allo sviluppo dell’intelligenza artificiale.

E’ di ieri infatti la notizia che un gruppo di aziende ed investitori della Silicon Valley, legati a Facebook e Google, hanno investito 1,000,000,000 (un miliardo!) di dollari in un centro di ricerca non-profit per l’avanzamento dell’intelligenza artificiale.
I suoi creatori e finanziatori, da una parte riconoscono che “E’ difficile financo immaginare quanto l’invenzione dell’intelligenza generale di livello umano possa beneficiare l’umanità, ed è equalmente difficile immaginare quanto essa la possa danneggiare se costruita o usata incorrettamente4.

Ma dall’altra essi definiscono il proprio ruolo come “sarà importante avere un’istituzione di ricerca guida che possa dare priorità ad un buon risultato finale per tutti rispetto ai suoi stessi interessi5. Tali soggetti si sentono, evidentemente, completamente a loro agio nell’immaginare in mano a pochi tecnocrati di successo – e la loro etica, interessi, visioni, e sogni d’immortalità – la governance mondiale della più grande sfida che l’umanità abbia mai affrontato.

Sfida che nell’arco di pochi o pochissimi decenni porterà, con ogni probabilità, a cambiamenti che non hanno precedenti nella storia dell’umanità, fin dalla nascita della vita biologica sul nostro pianeta.

—————————————————-

NOTE

1 La Roosevelt disse: “During a plenary session in the General Assembly, this young man tried to make a speech from the balcony on the subject of how incompetent the United Nations is to deal with the questions before it. How much better it would be if Mr. Davis would set up his own governmental organisation and start then and there a worldwide international government”

2http://www.rufoguerreschi.com/2007/08/29/a-un-blessed-but-democratic-world-constituent-assembly/

3Come disse egregiamente Richard Falk.

4From the OpenAI home page, on Dec 14th 2015: “It’s hard to fathom how much human-level AI could benefit society, and it’s equally hard to imagine how much it could damage society if built or used incorrectly.”

5From the OpenAI home page, on Dec 14th 2015: “… it’ll be important to have a leading research institution which can prioritize a good outcome for all over its own self-interest.”,

 

“Islamic” terrorism and western state-terrorism can only be reduced together

Today, there was an islamic terrorist massacre in Paris.

Aside from madness, what could the Paris Massacre terrorists, and those that support or strategize behind them, possibly have aimed to achieve?!

It can only be an increase of fear and hate among innocent civilian of 2 different religious faiths and cultures, that would lead to more war in Islamic states, and then to the coming to power of more fanatic irrational regimes that claim to represent true Islamic faith.

But more war in islamic states, with “collateral” massacres and injustices towards millions of islamic civilians, is unfortunately a goal that – for Cristian religious fanaticism and hate, political misjudgment or huge economic interests – has also been very actively promoted by some western private (oil and defense contractors) and governmental actors.

We can’t fight one without the other.

Welcome to Linear City 2.0, a social and human urban redevelopment concept

For my master thesis in Public Policy and Regional Planning at Rutgers University in 2000, I defined in fine detail an ethical vision I had in 1998 that convinced me to pursue that Master in that school: the technical, political and conceptual business plan for a LINEAR CITY (1.0), i.e. a large-scale intermodal urban corridor RE-development, heavily centered on public transport and light electric vehicles, to make cities social, human and ecologically sound. I even had full 3D animations done by myself with amazing detail:
www.linearcity.org

WELCOME TO LINEAR CITY 2.0

Fifteen years later – given all the advances in self-driving vehicles, and the fact that Linearcity that it will still take many years before they are authorized on the streets, and decades before they reach majority of cars – my Linearity concept could be amended by substituting all feeder systems to the main subway/train – which are in version 1.0 a mix of mixed-grade bus and automated guided buses (i.e. with driver!) – with pure self-driving small buses, but on a mix of separate-grade and mixed-grade. In some case, separate-grade may just be a preferential line well-marked on the asphalt, and sidewalk pedestrian warning, without physical separation.

Some comments on the Preamble of the Italian Internet “Bill of Rights”

Last July 2015, the Italian parliament approved, through a motion, an Italian Internet “Bill of Rights”. We greatly admire and support the motives of the drafters, many of which are friends, but we believe it necessary to highlight some serious shortcomings to its approach, starting with its Preamble.

PREAMBLE

It has fostered the development of a more open and free society.

This is very arguable. A large majority of digital rights activists and IT security and privacy experts would disagree that, overall, it has.

The European Union is currently the world region with the greatest constitutional protection of personal data, which is explicitly enshrined in Article 8 of the EU Charter of Fundamental Rights.

This is correct, although Switzerland may be better in some regards.Nevertheless, even such standards to date have not at all been able to stop widespread illegal and/or inconstitutional EU states bulk surveillance, until Snowden and Max Schrems came along. Furthermore, even if the US and EU states fully adhered to EU standards, it would significantly improve assurance for passive bulk surveillance, but it would do almost nothing for highly scalable targeted endpoint surveillance (NSA FoxAcid, Turbine, hacking Team, etc), against of tens and hundreds of thousands of high-value targets, such as activists, parliamentarians, reporters, etc.

Preserving these rights is crucial to ensuring the democratic functioning of institutions and avoiding the predominance of public and private powers that may lead to a society of surveillance, control and social selection.

“May” lead?! There is a ton of evidence available for the last 2 years that to a large extent we have been living for many years in a “society of surveillance, control and social selection.”

Internet … it is a vital tool for promoting individual and collective participation in democratic processes as well as substantive equality

Since it has emerged to be overwhelmingly a tool of undemocratic social control, it would be more correct to refer to its potential to “promoting individual and collective participation in democratic processes”, rather than a current actual fact.

The principles underpinning this Declaration also take account of the function of the Internet as an economic space that enables innovation, fair competition and growth in a democratic context.

By framing this at the end of the preamble, it makes it appear that privacy and civil rights needs are obstacles to innovation, fair competition and growth, which is not the case, as the Global Privacy as Innovation Network has been clearly arguing for over 2 years.

A Declaration of Internet Rights is crucial to laying the constitutional foundation for supranational principles and rights.

First, there have been about 80 Internet Bill of Rights approved by various stakeholders, including national legislative bodies. Second, a “declaration of rights” can very well be just smoke in the eyes, if those rights are not defined clearly enough and meaningful democratic enforcement is also enacted. There are really no steps towards proper “Supranational principles and rights”, and related enforcement mechanism, except a number of nations bindingly agreeing to them, similarly to the process that lead to creation of the International Criminal Court.

Richard Hawking on the great risks of the “default” scenarios for the future of AI

Richard Hawking, the great physicist, sees in the future of humanity like no one else. He sees our greatest risks related to the future of self-improving AI machines:

(1) Human exinction, if AI machines can be controlled at all. He said “Whereas the short-term impact of AI depends on who controls it, the long-term impact depends on whether it can be controlled at all”.

(2) Huge wealth [and power]  gaps, if AI machine owners will allow a fair distribution once these will take on all human labor. He said “If machines produce everything we need, the outcome will depend on how things are distributed.” Hawking continued, “Everyone can enjoy a life of luxurious leisure if the machine-produced wealth is shared, or most people can end up miserably poor if the machine-owners successfully lobby against wealth redistribution. So far, the trend seems to be toward the second option, with technology driving ever-increasing inequality.”

Is meaningful trustworthiness a requirement of Free Software “computing freedom”?

In this youtube video excerpt (minute 8.33-15.55) from Panel 2 of the Free and Safe in Cyberspace conference, that I organized 2 weeks ago, in which Richard Stallman and myself debate about IT trustworthiness and free software. The entire panel video is also available in WebM format here.

In such excerpt, Richard Stallman said that computing trustworthiness is a “practical advantage or convenience” and not a requirement for computing freedom. I opposed to that a vision by which the lack of meaningful trustworthiness turns inevitably the other four software freedoms into a disutility to their users, and to people with whom they share code. I suggest that this realization should somehow be “codified” as a 5th freedom, or at least very widely acknowledged within the free software movement.

How could the US government incentivize IT service providers to voluntarily and adequately provide compliance to lawful access?!

More news on Obama’s search for legislative or regulatory solution to lawful access to digital systems.

For some time now, the US government has been ever more often stating that there will not be a mandatory technical requirements to enable remote state lawful access, but that they expect provider to somehow come up autonomously with solutions that would allow for lawful access when needed by investigating agencies.

But any company that decided to come up with some techncial and organizational processes to do so, even with extremely effective safeguards for both the citizen and the investigating agency, would appear to be, and possibly actually be, less secure than competing services or devices that do not provide such access.

This problem could be solved if the US government provided very solid and reliable incentives to those that do, and do in a proper way, i.e., they comply to a minimum of citizen-accountable extreme safeguards, that guarantee both the user and the agency. The US government could approve some solidly enforceable policies that prescribe much higher personal economic and penal consequences for official of state agencies that are found searching or implanting vulnerabilities ONLY for high-assurance IT service providers that offer socio-technical systems to comply to government request, as certified by an independent international technically-proficient and accountable certification body. Such new policies would instead exclude IT service or device providers that do not.

To get 2 beans with one stone, such international body could also certify IT services and devices that offer meaningfully high-levels of trustworthiness, something that is direly missing today. One such certification body is being promote by the Open Media Cluster (that I lead), with the name of Trustless Computing Certification Initiative.

Le Dimissioni di Marino: “rule of law” contro “o’ Sitema”

Oggi si è dimesso Marino da sindaoc del Comune di Roma.

Nel 1992 con Tangentopoli, l’economia si fermò per 1-2 anni e quasi nessuno andò in galera. Nuovi partiti si formarono che per la quasi totalità continuarono come prima o peggio.

Oggi con Marino succede qualcosa di simile, ma senza nemmeno gli onori della cronaca, con media schierati a batteria su presunti pasti a scrocco del sindaco, invece di parlare dell’assalto strutturale ed ininterrotto per centinaia di milioni di euro alle casse del Comune.

Se ne deve concludere che chiunque provi anche solo a non compromettersi e prestarsi all’ “illegalità diffusa di alto livello orchestrata dalla politica”, nei limiti delle competenze di un amministratore, verrà accusato di “non fare”, pressato attraverso vari ostruzionismi finalizzato al peggioramento dei servizi, e manovre di stampa per far perdere consenso politico.

Si è provato ad eleggere magistrati con grande consenso politico come De Magistris, ma non c’è stato quasi niente da fare; come ha provato a fare qualcosa gli bloccavano i trasporti e la monnezza e lo isolavano con i media.

Si è provato con il professore indipendente dall’America, Marino, ma siccome non si partecipa a quello che a Napoli chiamo “o Sistema”, stesso trattamento. Dicono “non lega con i Romani”, perché tutti i media locali dicono che è un ladro nullafacente ed è semplicemente una persona seria.

L’unico modo di uscirne sarà quando un sindaco verrà eletto con il mandato chiaro e centrale – supportato da un partito che con credibilità rispetto alla sua storia – di ripulire il malaffare di strutturale e in grande scala, e NON le ricevutine dei pranzi. Solo allora i media si allineeranno a spiegare ai romani che se i servizi non funzionano e le casse piangono è per il malaffare e non per chi cerca di contrastarlo. Non so se ci sia già in Italia un partito o una forza politica così …

A Proposed Solution to Wikimedia funding problem …

… without introducing any undemocratic bias:

Introduce contextual ads made exclusively of product/service comparisons made by  democratically-controlled consumer organizations. In Italy for example there is Altroconsumo org with 100s of thousands of members which regularly produces extensive comparative reports.

In practice: for each new report that comes out, a request is made to the companies producing the product/service in the top 30% to sponsor it publishing inside Wikimedia portals.
Such formula could be extended to Wikimedia video, generating huge funds, arguably without any. Proceed are shared among Wikimedia and the consumer org.

(originally written in 2011, and sent to Jimmy Whale, which found it interesting)

“f no values-based standards exist for Artificial Intelligence, then the biases of its manufacturers will define our universal code of human ethics. But this should not be their cross to bear alone. It’s time to stop vilifying the AI community and start defining in concert with their creations what the good life means surrounding our consciousness and code.”

http://mashable.com/2015/10/03/ethics-artificial-intelligence/?utm_cid=mash-com-Tw-tech-link

“Unabomber with flowers”. May it be our best option to stave off AI superintelligence explosion?

There are many ways to try to prevent catastrophic AI developments by actively getting involved as a researcher, political activist or entrepreneur. In fact, I am trying to do my part as a Executive Director of the Open Media Cluster.

But maybe the best thing we can do to help reduce chances of the catastrophic risks of artificial super-intelligence explosion (and other existential risks) become a “Unabomber with flowers“.

By that I mean, we could hide out in the woods, as the Unabomber did, to live in modern off-grid eco-villages somewhere. But, instead of sending bombs to those most irresponsibly advancing general Artificial Intelligence, we’d send them flowers, letters and fresh produce, and invitations for a free travel in the woods.

Here’s what the  wrote in the Unabomber wrote in his manifesto “Industrial Society and Its Future”, published by the New York Times in 1995:  

173. If the machines are permitted to make all their own decisions, we can’t make any conjectures as to the results, because it is impossible to guess how such machines might behave. We only point out that the fate of the human race would be at the mercy of the machines. It might be argued that the human race would never be foolish enough to hand over all the power to the machines. But we are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power. What we do suggest is that the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines decisions. As society and the problems that face it become more and more complex and machines become more and more intelligent, people will let machines make more of their decision for them, simply because machine-made decisions will bring better result than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won’t be able to just turn the machines off, because they will be so dependent on them that turning them off would amount to suicide.

My wife Vera and my dear friend Beniamino Minnella surely think so.

IT security research needs for artificial intelligence and machine super-intelligence

(originally appeared on Open Media Cluster website on July 7th 2015)

On Jan 23rd 2015, nearly the entire “who’s who” of artificial intelligence, including the leading researchers, research centers, companies, IT entrepreneurs – in addition to what are possibly the leading world scientists and IT entrepeneurs – have signed Open Letter Research priorities for robust and beneficial artificial intelligence with an attached detailed paper (we’ll refer to both below as “Open Letter”).

In this post, we’ll look at such Open Letter and ways in which its R&D priorities in the areas of IT security may crucially need to be corrected, and “enhanced” in future version.

We’ll also look at the possibility that short-term and long-term R&D needs of artificial intelligence “(“AI”) and information technology (“IT”) – in terms of security for all critical scenarios – may become synergic elements of a common “short to long term” vision, producing huge societal benefits and shared business opportunities. The dire short-term societal need and market demand for radically more trustworthy IT systems for citizens privacy and security and societal critical assets protection, can very much align – in a grand strategic cyberspace EU vision for AI and IT – with the medium-term market demand and societal need of large-scale ecosystems capable to produce AI systems that will be high-performing, low-cost and still provide adequately-extreme levels of security for AI critical scenarios.

But let’s start from the state of the debate on the future of AI, machine super-intelligence, and the role of IT security.

In recent years, rapid developments in AI specific components and applications, theoretical research advances, high-profile acquisitions from important global IT giants, and heart-felt declaration on the  dangers of future AI advances from leading global scientists and entrepreneurs, have brought AI to the fore as both (A) a key to economic dominance in IT, and other business sectors, as well as (B) the fastest emerging existential risk for humanity in its possible evolution into uncontrolled machine super-intelligence.

Google, in its largest EU acquisition this year acquired for 400M€ a global AI leader, DeepMind; already invested by Facebook primary initial investors Peter Thiel and Elon Musk. Private investment in AI has been increasing 62% a year, while it is not known – but presumably very large and fast increasing – the level of secret investments by multiple secretive agencies of powerful nations, such as the NSA, in a possibly already-started winner-take-all race to machine super-intelligence among public and private actors.

Global AI experts on average estimate that there is a 50% chance to achieve human-level general artificial intelligence by 2040 or 2050, while not excluding significant possibilities that it could be reached sooner. Such estimates may be strongly biased towards later dates because: (A) there is an intrinsic interest in those that are by far the largest investors in AI – global IT giants and USG – to avoid risking a major public opinion that a major political; (B) As it has happened for surveillance program and technologies of Five Eyes countries, it plausible or probable that huge advancements in AI capabilities and programs may have already happened but successfully kept hidden for many years and decades, even while involving large numbers of people.

Many and increasing numbers of experts believe that progress beyond such point may become extremely rapid, in a sort of “intelligence explosion”, posing grave questions on humans ability to control it at all. (See Nick Bostrom TED presentation). Very clear and repeated statements by Stephen Hawking (the most famous scientist alive), by Bill Gates, by Elon Musk (main global icon of enlightened tech entrepreneurship), By Steve Wozniak (co-founder of Apple), agree on the exceptionally grave risks posed by uncontrolled machine super-intelligence.

Elon Musk, shortly after having invested in DeepMind, even declared, in an erased but not retracted comment:

“The pace of progress in artificial intelligence (I’m not referring to narrow AI) is incredibly fast. Unless you have direct exposure to groups like Deepmind, you have no idea how fast-it is growing at a pace close to exponential. The risk of something seriously dangerous happening is in the five-year timeframe. 10 years at most. This is not a case of crying wolf about something I don’t understand.”

I am not alone in thinking we should be worried. The leading AI companies have taken great steps to ensure safety. The recognise the danger, but believe that they can shape and control the digital superintelligences and prevent bad ones from escaping into the Internet. That remains to be seen…”

Such Open Letter is an incredibly important and well-thought out, and important to increase the chance that the overall impact of AI in coming decades – large in the medium term and huge in the long-term by all account – will be in accordance to humanities values and priorities. Nonetheless, such document comes with what we believe to be potentially gravely erroneous assumptions about the current state-of-the-art and R&D directions in IT security of high-assurance systems, which in turn would potentially completely undermine its verification, validity and control. 

In general, the such Open Letter overestimate the levels of trustworthiness, measurability, the at-scale costs, of existing and planned highest-assurance low-level computing systems and standards. 

More in detail, here are line by line suggestions to the Short Term Research Priorities – 2.3.3 Security section, from page 5: 

2.3.3   Security

Security research can help make AI more robust.

A very insufficiently-secure AI system may be greatly “robust” in the sense of business continuity, risk management and resilience, but still be extremely weak in safety or reliability of control. This outcome may sometimes be aligned with the AI sponsor/owner goals – and those of other third parties such as state security agencies, publicly or covertly involved – but be gravely misaligned  to chances to maintain a meaningful democratic and transparent control, i.e. having transparent reliability about what the system, in actuality, is set out to do and who, in actuality, controls it.

Much more important than “robustness”, adequate security is the most crucial foundation for AI safety and actual control in the short and long terms, as well as a precondition for verification and validity. 

As AI systems are used in an increasing number of critical roles, they will take up an increasing proportion of cyber-attack surface area. It is also probable that AI and machine learning techniques will themselves be used in cyber-attacks.

There is a large amount of evidence that many AI techniques have long been and are [1] currently being used by the most powerful states intelligence agencies, to attack – often in contrast with national or international norms – end-users and IT systems, including IT systems using AI. As said above, while it is not known the levels of investment of public agencies of powerful nations such as the NSA, is presumably very large and fast increasing,  in a possibly already started race against among public and private actors. The distribution of such finding aims most likely will follow the current ratio of tens of times more resources to offensive R&D rather than defensive R&D.

Robustness against exploitation at the low-level is closely tied to verifiability and freedom from bugs. 

This is a correct although partial. Especially for use in critical and ultra-critical use cases, which will become more and more dominant.

   It is better to talk about auditability in order not get confused with (formal) IT verification. It is crucial and unavoidable to have complete public auditability of all critical HW, SW and procedural components involved in an AI systems life-cycle, from certification standards setting, to CPU design, to fabrication oversight. In fact, since 2005 US Defense Science Board has highlighted how “Trust cannot be added to integrated circuits after fabrication” as vulnerabilities introduced during fabrication can be impossible to verify afterwards. Bruce Schneier, Steve Blank, and Adi Shamir,  among others, have clearly said there is no reason to trust CPUs and SoCs (design and fabrication phases). No end-2-end IT system or standards exist today that provide such complete auditability of critcal components. 

   “Freedom from bugs” is a very improper term as it excludes voluntarily introduced vulnerabilities, or backdoors, and it should clearly differentiate between critical and non-critical bugs. Vulnerabilities may be accidental (bug) or voluntary (backdoor). It is often impossible to prove that a vulnerability was introduced voluntarily and not accidentally. We should talk of “Freedom from critical vulnerabilities
It is impossible, and most probably will remain so, to ensure perfectly against critical vulnerabilities, given the socio-technical complexity of IT socio-technical systems even if simplified by 10 or 100 times, and with radically higher levels of auditing relative to complexity.
Nonetheless, it remains
extremely crucial and fundamental that adequate research could device ways to achieve sufficiently-extreme level confidence about “freedom from critical vulnerabilities” through new paradigms to achieve sufficient user-trustworthiness that sufficient intensity and competency of engineering and auditing efforts relative to complexity have been applied, for all critical software and hardware components that are actually running on the involved device. No system or standard exist today to systematically and comparatively assess – for such target levels of assurance for a given end-2-end computing service, and its related life-cycle and supply-chain.  

As stated above, all AI systems in critical use cases – and even more crucially those in advanced AI system that will soon be increasingly approaching machine super-intelligence – will need to be so robust in terms of security so such as extent that they are resistant against multiple extremely-skilled attackers willing to devote cumulatively even tens or hundreds of millions of Euros to compromise at least one critical components of the supply chain or life-cycle, through legal and illegal subversion of all kinds, including economic pressures; while having high-level of plausible deniability, low risk of attribution, and (for some state actors) minimal risk of legal consequences if caught.

In order to reduce substantially this enormous pressure, it may be extremely useful to research socio-technical paradigms by which sufficiently-extreme level of AI systems user-trustworthiness can be achieved, while at the same time transparently enabling due legal process cyber-investigation and crime prevention. The possible solution of such dichotomy would reduce the level of pressure by states to subvert secure high-assurance IT systems in general, and possibly – through mandatory or voluntary standards international lawful access standards – improve the ability of humanity to conduct cyber-investigations on the most advanced private and public AI R&D programs.

For example, the DARPA SAFE program aims to build an integrated hardware-software system with a flexible metadata rule engine, on which can be built memory safety, fault isolation, and other protocols that could improve security by preventing exploitable flaws [20]. Such programs cannot eliminate all security flaws (since verification is only as strong as the assumptions that underly the specification), but could significantly reduce vulnerabilities of the type exploited by the recent “Heartbleed bug” and “Bash Bug”.

There is a need to avoid the risk of relying for guidance on high-assurance low-level systems standard/platform projects from defense agencies of powerful nations, such as the mentioned DARPA SAFE, NIST, NSA Trust Foundry Program, DARPA Trust in Integrated Circuits Program, when it is widely proven that their intelligence agencies (such as NSA) have gone to huge length to surreptitiously corrupt technologies and standards, even those that are overwhelmingly used internally in relatively high-assurance scenarios.

Such systems could be preferentially deployed in safety-critical applications, where the cost of improved security is justified.

The cost of radically more trustworthy low-level system for AI could become very comparable to those of current corporate-grade security IT systems, mostly used as standard in AI systems development. Those costs differentials could possibly be reduced to being insignificant through production at scale, and open innovation models to drive down royalty costs. For example, hardware parallelization of secure systems and lower unit costs, could make so that adequately secure systems could compete or even out compete in cost and performance those other generic systems. (The emerging non-profit User Verified Social Telematics consortium, for example, show the possibility of creating sufficiently-secure general-purpose computing systems running at 1-300Mhz with a cost made of cost of production (few tens of euros depending on quantity), and overall royalty costs of only 30% of the end-user cost.)

At a higher level, research into specific AI and machine learning techniques may become increasingly useful in security. These techniques could be applied to the detection of intrusions [46], analyzing malware [64], or detecting potential exploits in other programs through code analysis [11].

There is a lot of evidence to show that R&D investment on solutions to defend devices from the inside (that assume failure in intrusion prevention), could become end up increasing the attack surface if those systems life-cycle are not themselves subject to the same extreme security standards as the low-level system on which they rely upon. Much like antivirus tools, password storing application and other security tools are often used a ways to get directly to a user or end-point most crucial data. Recent scandal of NSA, Hacking Team, JPMorgan show the ability of hackers to move inside extremely crucial system without being detected, possibly for years. DARPA high-assurance program highlight how about 30% of vulnerabilities in high-assurance systems are introduced by internally security products.[2]

It is not implausible that cyber attack between states and private actors will be a risk factor for harm from near-future AI systems, motivating research on preventing harmful events.

Such likelihood is clearly higher than “not implausible”. It is not correct to say that it “will be a risk factor” as it is already a risk factor and at least one of the parties in the such cyber attacks, powerful states, are now extensively using and expectedly aggressively advancing AI tools.

As AI systems grow more complex and are networked together, they will have to intelligently manage their trust, motivating research on statistical-behavioral trust establishment [61] and computational reputation models [70].

Interoperability framework among AI systems, and among AI and IT systems, will need effective independent ways to assess the security of the other system. As stated above, current comparative standards are lacking so comprehensiveness and depth to make it impossible to compare the security of a given system.

Ultimately, it may be argued that IT security is about the nature of the organizational processes involved and the intrinsic constrains and incentives critically involve in individual within such organizations. Therefore, the most critical security factor to be researched, for critical AI systems in the short and long term, is probably will be the technical proficiency and citizen accountability of the organizational processes, that will govern the setting of key AI security certification standards or system, and the socio-technical systems, that will be deployed to ensure extremely effective and citizen-accountable oversight processes of all critical phase in the supply-chain and operational life-cycle of the AI system.

The dire short- term societal need and market demand for radically more trustworthy IT systems for citizens privacy and security and societal critical assets protection, can very much align in a grand strategic cyberspace EU vision to satisfy – in the medium and long-term – both the huge societal need and great economic opportunity of creating large-scale ecosystems able to produce AI systems that will be high-performing, low-cost and still provide adequately-extreme levels of security for AI critical scenarios.

NOTES

[1] See the National Security Analysis Center or the capabilities offered by companies like Palantir

[2] https://youtu.be/3D6jxBDy8k8?t=4m20s

The robots aren’t taking our jobs; they’re taking our leisure

But what about the bounty of digital technology that is in evidence all around us? Almost 30 years ago, the great economist Robert Solow quipped, “You can see the computer age everywhere but in the productivity statistics.”

An answer to the riddle might be that digital technology has transformed a handful of industries in the media/entertainment space that occupy a mindshare that’s out of proportion to their overall economic importance. .

http://www.vox.com/2015/7/27/9038829/automation-myth?utm_campaign=vox&utm_content=chorus&utm_medium=social&utm_source=twitter

Blaming China for cyber attacks without any public evidence creates highly-perverse dynamics

Blaming China for cyber attacks without any public evidence creates highly-perverse dynamics: (1) breached entity, instead of paying in liability/blame for lack of security, can turn itself into victim of act of war; (2) increases support for requests by defense  agencies/contractors for huge funds and anti-privacy anti-privacy legislation; (3) any expert or media who challenges misattribution becomes enemy of the state; (4) no serious investigation in who really behind attacks, why they did it, and why they succeeded; (5) retaliation from China can just make all of this escalate.

Please, every expert go out there and challenge the actual evidence (and lack thereof) of China government responsibility in the attacks!

A definition of “Constitutionally-meanigful levels of trustworthiness” in IT systems

A proposed definition of “Constitutionally-meanigful levels of trustworthiness” in IT systems

An IT system (or more precisely a end-2-end computing service or experience) will be said to have “constitutionally-meaningful levels of trustworthiness” when its confidentiality, authenticity, integrity and non-repudiation is sufficiently high to make its use – by ordinary, active and “medium-value target” citizens alike –rationally compatible to the full and effective Internet-connected exercise of their core civil rights, except for voting in governmental elections.  In concrete terms, it defines an end-2-end computing experience that warrants extremely well-placed confidence that the cost and risks for an extremely-skilled attacker to remotely perform continuous or pervasive comprimization substantially exceed the following: (1) for comprimization of a single user, the tens of thousands of euros, and the significant discoverability, such as those associated with enacting such level of abuse through on-site, proximity-based user surveillance, or non-scalable remote endpoint techniques, such as NSA TAO; (2) For the comprimization of the entire supply chain or lifecycle, the tens of millions of euros and significant discoverability, that are reportedly typically sustained by advanced actors, for high-value supply chains, through legal and illegal subversions of all kinds, including economic pressures.”

Motives of the Hacking Team hack may have much in common with those that broughtin 1903 the British Mr Maskelyne – and possibly its UK corporate/state sponsors – to hack Marconi’s radio telegraph in 1903 …

… to establish their tech/service as the “secure” remote communications of choice for global corporations and governments:

Maskelyne followed his trick with an even bigger showstopper. In June 1903, Marconi was set to demonstrate publically for the first time in London that morse code could be sent wirelessly over long distances. A crowd filled the lecture theatre of the Royal Institution while Marconi prepared to send a message around 300 miles away in Cornwall. The machinery began to tap out a message, but it didn’t belong to the Italian scientist.

“Rats rats rats rats,” it began. “There was a young fellow of Italy, who diddled the public quite prettily …” Maskelyne had hijacked the wavelength Marconi was using from a nearby theatre. He later wrote a letter to the Times confessing to the hack and, once again, claimed he did it to demonstrate the security flaws in Marconi’s system for the public good.

Of course cable could be undetectably be “sniffed” then as fiber cable can be sniffed today …