My blog

Why I think Schmidhuber’s conclusion of non-interfering AI is wrong

2016-08-11T00:00:00+02:00

I really enjoyed watching Prof. Jürgen Schmidhuber’s talk about future of AI. The topic of AI becoming Terminator-style is an evergreen and as a person who actually works in research I start to get quite annoyed about these futurology predictions. I think they are similarly annoying as quantum physics interpretations of The Secret movie - mathematical truths twisted by knaves to make a trap for fools (yes I like Kipling ;-)

His argument is that super AI will not be concerned with humans that much as we are not concerned with for example ants, because our goals are different. The only time we really become aware of ants and think about them is when they come inside the house and annoy us in the kitchen. Super AI would be somewhat similar to this.

That makes a lot of sense. The problem is that this AI would have a lot of friction points with humans, because it would compete for scarce resources. Machines that we build today are built out of relatively scarce materials, and presumably one of the goals of an AI like this would be to spread itself (like life). Which it couldn’t do if it needs scarce metallic resources and needs to compete with humans for them. Once it could move to a different, non-competitive place, like the Moon, where it could find these materials, there’s no reason for interference.

But anyway, I think discussions on these topics are still quite irrelevant today, because all we can do today is to build some statistical models that have become a bit more advanced than in the past.

Just my two cents.

Protocol buffers - python tips

2016-07-21T00:00:00+02:00

Protocol buffers (protobufs) are well known Google’s mechanism for a language-neutral, platform-neutral, extensible mean for serializing structured data.

Basically, it’s exchange format that is smaller than JSON/XML/whatever because it is saved in binary. And you can use code generators to make it very simple to use it in different languages.

To make it to work quickly, just get the binary release from official relases and put the protoc command to /usr/bin/protoc. Once you write your own .proto spec file, like the addressbook in examples you can compile it with

$ protoc --python_out=. *.proto

The docs for how to create the .proto specs are well documented on their website, so no need to explain it here. Maybe just a few things to keep in mind when you write your protocol headers:

Extending protobufs: - you must not change the tag numbers of any existing fields. - you must not add or delete any required fields. - you may delete optional or repeated fields. - you may add new optional or repeated fields but you must use fresh tag numbers (i.e. tag numbers that were never used in this protocol buffer, not even by deleted fields).

Currently there are two active versions:

The proto2 version is still actively supported, probably mainly because I find the proto3 kind of sucks :-) (one thing I didn’t like it doesn’t have support for default values. As it turns out, I’m not the only one who doesn’t like it but it has reasons ).

In python I struggled for an hour to find a way how to create human readable .prototxt files. All you have to do is (extending the python examples add_person.py and list_people.p )

# import this
from google.protobuf import text_format
# and then use this to read
text_format.Parse(f.read(), address_book)
# and this to write
text_format.MessageToString(address_book)

Enjoy.

Sphinx - python tips

2016-07-20T00:00:00+02:00

I recently started to educate myself more in Python. I decided that from time to time I’m going to put here some tips and tricks, things that I’ve come across that I think are interesting. These are comments from a person who has quite a lot of experience with other languages, such as Java, PHP and Matlab. I’ll put up links to these articles on twitter under hashtag #pythontips, so if you like these you can just follow the hashtag.

Sphinx

I started to work on a new project and since I have quite a free hand at what I’m doing, I decided to do things right. One of the most important things in any project is its documentation.

I searched for what would be nice in Python, actually there’s a huge list of different documentation generator systems on wikipedia Searching on the web I found Sphinx, seemed quite nice so I gave it a try.

In my conda installation all I had to do is:

$ conda install sphinx
# and then follow this set-up tutorial
$ sphinx-quickstart

It is going to ask you bunch of questions - if you want to generate docs from code, don’t forget to allow autodoc

Useful links:

Here is a link to tutorial: https://codeandchaos.wordpress.com/2012/07/30/sphinx-autodoc-tutorial-for-dummies/
How to write code for .rst files: https://pythonhosted.org/an_example_pypi_project/sphinx.html
Use markdown instead of rst: http://searchvoidstar.tumblr.com/post/125486358368/making-pdfs-from-markdown-on-readthedocsorg-using (I didn’t get that to work properly yet, but others seem to have)
All kinds of options: http://www.sphinx-doc.org/en/stable/domains.html

I set up my project with following structure

├── data
├── docs        # here is Sphinx documentation
│   ├── _build
│   ├── conf.py
│   ├── index.rst
│   ├── make.bat
│   ├── Makefile
│   ├── _static
│   └── _templates
├── Makefile
├── NOTES.md
├── README.md
├── requirements.txt
├── src              # my main code
├── temp
└── tests

conf.py has all of the Sphinx config, the one that you setup in the spinhx-quickstart. Find sys.path.insert and add sys.path.insert(0, os.path.abspath('../src'))

My main project Makefile has these scripts:

# I wanted to have it called docs, but that confuses make command
# since there is a directory there already called docs - these two have to be distinct.
doc: 
	# remove all old rst files except index.rst
	find ./docs -name '*.rst' ! -name 'index.rst' -type f -exec rm -f {} \;
	# generate new .rst files for docs
	sphinx-apidoc -l -F --module-first -o docs src
	# vrrrrruuummm!!! generate docs
	cd docs; make html;

showdoc:
	chromium-browser docs/_build/html/index.html

That’s it, I just type make doc and then make showdoc and the documentation is born. Voila!

Human learning is way more efficient in power consumption… for now.

2016-06-20T00:00:00+02:00

When I was eating dinner a simple thought occurred to me: is machine learning more power efficient than human learning… from scratch?

A neural network called AlexNet was trained between five and six days on two GTX 580 3GB GPUs which have 363W consumption.

So simply calculating,

$W_{GPU} = (5+6)\times 24 \times 363 = 95832\text{Wh}.$

I found some site that describes recommended energy intakes for newborn babies. After some funny discussion we came to conclusion with my mom that I might’ve been able to perform ImageNet when I would have been 3 years old, so if I take that data, then

$W_{human} = 550\times 6\times 30+700\times 6\times 30+1000\times 2\times 12 = 249000\text{cal} = 289.587 \text{Wh}.$

By comparing these I got to conclusions that humans are about 300 times more power efficient in learning from scratch. Of course, I took an old version of GPU training so that might differ nowadays (I didn’t find other GPU usages quickly, but I knew I would find them in Alex’s paper. Maybe someone can calculate this for other nets?).

Let’s assume that energy efficiency has the same exponential as Moore’s law so that it halves every 18 months. The GTX 580 launched on 9 November 2010 so it’s almost six years now, and the consumptions should be at the level of 363\times 2^{-4} = 22W. With this consumption it would take in total 6kW. By projecting these numbers human level power efficiency should be achieved by year 2022 - 6 years from now. And with better algorithms it’s probably going to be even sooner. Isn’t this exciting?

Predpovede na rok 2016

2015-12-23T00:00:00+01:00

Chcem sa zahrať na futurológa :-) tak skúsim predpovedať čo sa stane v roku 2016 (hlavne v technológiach):

pomocou deep learningu sa dostanú do áut prvé zariadenia ktoré budú robustne vykonávať jednoduché riadenie, napr. na ďiaľniciach. Pomôcť tomu môžu projekty ako http://www.technologyreview.com/view/544926/ai-machine-learns-to-drive-using-crowdteaching/ alebo https://www.youtube.com/watch?v=KTrgRYa2wbI hoci Tesla si myslí že je taký systém príliš jednoduchý http://www.techinsider.io/elon-musk-responds-to-george-hotz-bloomberg-story-2015-12
čoraz väčšie množstvo firiem začne používať AI, zdroje ktoré uvoľňuje FB a Google sú úplne neskutočné, napr. nedávno ich TensorFlow
budú sa inkrementálne zlepšovať modely sietí, zlepšovať hardvér, zväčšovať hĺbka, ale nejaký breakthrough v podobe zaujímavého iného prístupu (ako napr. spôsobil AlexNet) ešte nenastane, nakoľko sa možnosti toho čo sa dá s týmito modelmi ešte všetko robiť nevyčerpalo.
bude zdvojnásobné množstvo qubitov na kvantových počítačoch
vláda U.S. a U.K. sa bude ďalej snažiť obmedzovať šifrovanie na internete pomocou vpašovávania rôznych debilných zákonov
prvýkrát vznikajú systémy ktoré sú schopné robustného mass-surveillance, čo je hrozivé, zvlášť pri rôznych iniciatívach typu https://www.youtube.com/watch?v=lHcTKWiZ8sI
začnú vznikať rôzne startupy okolo virtuálnej reality, využije toho spočiatku porno priemysel, hry, a časom prídu aj inžinierske nástroje (stavbárina a pod.)
po prezidentských voľbách 2016 v USA praskne startupová bublina, podobne ako dot-com bublina padla v čase po prezidentských voľbách v roku 2000