Jupyterjones - Productivity Please !!!

Note to self: debugging

always remember to remove the Log.d(”Well fuck you too”) lines and the variables named “wtf” and “whhhyyyyyyy” before pushing anything to git

Depixellation? Or hallucination?

There’s an application for neural nets called “photo upsampling” which is designed to turn a very low-resolution photo into a higher-res one.

This is an image from a recent paper demonstrating one of these algorithms, called “PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models”

It’s the neural net equivalent of shouting “enhance!” at a computer in a movie - the resulting photo is MUCH higher resolution than the original.

Could this be a privacy concern? Could someone use an algorithm like this to identify someone who’s been blurred out? Fortunately, no. The neural net can’t recover detail that doesn’t exist - all it can do is invent detail.

This becomes more obvious when you downscale a photo, give it to the neural net, and compare its upscaled version to the original.

As it turns out, there are lots of different faces that can be downscaled into that single low-res image, and the neural net’s goal is just to find one of them. Here it has found a match - why are you not satisfied?

And it’s very sensitive to the exact position of the face, as I found out in this horrifying moment below. I verified that yes, if you downscale the upscaled image on the right, you’ll get something that looks very much like the picture in the center. Stand way back from the screen and blur your eyes (basically, make your own eyes produce a lower-resolution image) and the three images below will look more and more alike. So technically the neural net did an accurate job at its task.

A tighter crop improves the image somewhat. Somewhat.

The neural net reconstructs what it’s been rewarded to see, and since it’s been trained to produce human faces, that’s what it will reconstruct. So if I were to feed it an image of a plush giraffe, for example…

Given a pixellated image of anything, it’ll invent a human face to go with it, like some kind of dystopian computer system that sees a suspect’s image everywhere. (Building an algorithm that upscales low-res images to match faces in a police database would be both a horrifying misuse of this technology and not out of character with how law enforcement currently manipulates photos to generate matches.)

However, speaking of what the neural net’s been rewarded to see - shortly after this particular neural net was released, twitter user chicken3gg posted this reconstruction:

Others then did experiments of their own, and many of them, including the authors of the original paper on the algorithm, found that the PULSE algorithm had a noticeable tendency to produce white faces, even if the input image hadn’t been of a white person. As James Vincent wrote in The Verge, “It’s a startling image that illustrates the deep-rooted biases of AI research.”

Biased AIs are a well-documented phenomenon. When its task is to copy human behavior, AI will copy everything it sees, not knowing what parts it would be better not to copy. Or it can learn a skewed version of reality from its training data. Or its task might be set up in a way that rewards - or at the least doesn’t penalize - a biased outcome. Or the very existence of the task itself (like predicting “criminality”) might be the product of bias.

In this case, the AI might have been inadvertently rewarded for reconstructing white faces if its training data (Flickr-Faces-HQ) had a large enough skew toward white faces. Or, as the authors of the PULSE paper pointed out (in response to the conversation around bias), the standard benchmark that AI researchers use for comparing their accuracy at upscaling faces is based on the CelebA HQ dataset, which is 90% white. So even if an AI did a terrible job at upscaling other faces, but an excellent job at upscaling white faces, it could still technically qualify as state-of-the-art. This is definitely a problem.

A related problem is the huge lack of diversity in the field of artificial intelligence. Even an academic project with art as its main application should not have gone all the way to publication before someone noticed that it was hugely biased. Several factors are contributing to the lack of diversity in AI, including anti-Black bias. The repercussions of this striking example of bias, and of the conversations it has sparked, are still being strongly felt in a field that’s long overdue for a reckoning.

Bonus material this week: an ongoing experiment that’s making me question not only what madlibs are, but what even are sentences. Enter your email here for a preview.

My book on AI is out, and, you can now get it any of these several ways! Amazon - Barnes & Noble - Indiebound - Tattered Cover - Powell’s - Boulder Bookstore

The 21 Card Trick created in Python.

See how it works and read a little more about it here: [x]

Feel free to ask any questions you may have. :)

When 25,000 little dice are agitated in a cylinder, they form into neat concentric circles

Behold the magic of compaction dynamics. Scientists from Mexico and Spain dumped 25,000 tiny dice (0.2 inches) into a large clear plastic cylinder and rotated the cylinder back and forth once a second. The dice arranged themselves into rows of concentric circles. See the paper and the videos here.

https://boingboing.net/2017/12/05/when-25000-little-dice-are-ag.html

Geometry at work: Maxwell, Escher and Einstein

Maxwell’s diagram

from the 1821 “A Philosophical Magazine”, showing the rotative vortexes of electromagnetic forces, represented by hexagons and the inactive spaces between them

The impossible cube

invented in 1958, as an inspiration for his Belvedere litography.

Geometry of space-time

The three dimensions of space and the dimension of time give shape to the fourth, that of space-time.

How to draw a regular pentagon [x]

The Kakeya Problem

Some time ago I (briefly) mentioned that, along with two other students, I was taking a reading course this semester with Dmitriy Bilyk. It hasn’t quite gone in the direction we were initially expecting, but one of our long detours has been an extended sequence of readings around the the Kakeya conjecture. As far as I know, the Kakeya problem (different from the Kakeya conjecture; more on that later) was the first question that fell respectably under the purview of geometric measure theory. So if nothing else, it is interesting from a historical perspective as a question that kickstarted a whole new field of mathematics.

Okay, but what is the Kakeya problem?

As originally posed, it goes like this: given a disk with diameter 1, it is possible to place down a line segment of length 1 into the set, and rotate it (continuously) an entire 180 degrees. But this is not the smallest-area set for which this kind of rotation is possible:

(all pictures in the post are from the Wikipedia page, which is really good)

This set has area $\tau/16$, half that of the circle. So the question naturally becomes: what is the area of the smallest set that allows this kind of rotation?

The answer is known, and it is…

…

zero, basically.

It’s remarkable, but it’s true: you can construct arbitrarily small sets in which you can perform a 180-degree rotation of a line segment! One way to do it goes like this: in the picture above, we reduced the area of the circle by squeezing it until it developed three points. If we keep squeezing to get more points, then the solid “middle” becomes very small, and the tendrils get very thin, so the area keeps decreasing. However, we can still take a line segment and slowly, methodically, shift it back and forth between the set’s pointy bits, parallel-parking style, to eventually get the entire 180 degrees of rotation.

——

You can’t actually get a set with zero area to work. But the reason for that seemed more like a technicality than something actually substantial. So people changed the problem slightly to get rid of those concerns. Now, instead of trying to get a set where you can rotate a line segment through all 180 degrees, you just have to have a set where you can find a line segment in every direction. The difference being that you don’t need to guarantee any smooth way to “move between” these line segments.

Sets that work for the modified problem are often called Kakeya sets (although some people reserve that for the rotation problem and use Besicovitch sets for the modified one). And indeed, there are Kakeya sets which actually have zero area.

The details involved with going from “arbitrarily small” to “actually zero” are considerable, and we won’t get into them here. The following is a simplification (due to Perron) of Besicovitch’s original construction for the “arbitrarily small” case. We take a triangle which clearly has ½ of the directions the needle might possibly take, and split it up into several pieces in such a way that no directions are lost. Then we start to overlap those pieces to get a set (that still has segments in all the same directions) with much smaller area:

We can do this type of construction twice (one triangle “downward-facing” and the other “left-facing”) and then put those two sets together, guarantee that we get all of 180 degrees of directions.

This Besicovitch-Perron construction, itself, only produces sets which are “arbitrarily small”, but was later refined to go all the way to zero. Again, the technicalities involved in closing that gap are (much) more than I want to talk about now. But the fact that these technicalities can be carried out with the Besicovitch-Perron construction is what makes it “better” than the usual constructions for the original Kakeya problem.

——

I should conclude with a few words about the Kakeya conjecture, since I promised them earlier :)

Despite the essentially-solved status of these two classical Kakeya problems, there is at least one big question still left open. It is rather more technical than the original ones, and so doesn’t get a lot of same attention, but I’d like to take a stab at explaining what’s still current research in this sphere of ideas.

Despite the fact that Kakeya sets can be made to be “small” in the sense of measure, we still intuitively want to believe these sets are “big”. There are many ways we can formalize largeness of sets (in $\Bbb R^n$, in particular) but the one that seems to be most interesting for Kakeya things is the notion of Hausdorff dimension. I won’t define the term here, but if you’ve ever heard someone spouting off about fractals, you’ve probably heard the phrase “Fractals have non-integer dimension!”. This is the notion of dimension they’re talking about.

It is known that Kakeya sets in the plane have Hausdorff dimension 2, and that in general a Kakeya set in $n>2$ dimensions has Hausdorff dimension at least $\frac{n}{2}+1$. The proofs of these statements are… difficult, and the general case remains elusive.

One thing more: you can also formulate the Kakeya conjecture in finite fields: in this setting having “dimension $n$” in a vector space over the field $\Bbb F_q$ means that you have a constant times $q^n$ number of points in your set. Wolff proposed this “technicality-free” version in 1999 as a way to study the conjecture for $\Bbb R$. And indeed, a lot of the best ideas for the problem in $\Bbb R$ have come from doing some harmonic analysis on the ideas originally generated for the finite field case. 

But then in 2008 Zeev Dvir went and solved the finite field case completely. Which on one hand is great! But on the other hand, Dvir’s method definitely can’t be finessed to work in $\Bbb R$ so we still have work to do :P

——

Partially I wanted to write about this because it’s cool in its own right, but I must admit that my main motivation is a little more pragmatic. There was a talk at SEICCGTC 2017 which showed a surprising connection between the Kakeya problem and a certain combinatorial game. So if you think these ideas are at all interesting, you may enjoy reading the next two posts in this sequence about that talk.

[ Post 1 ] [ Next ]

Eclipse Across America

August 21, 2017, the United States experienced a solar eclipse! 

An eclipse occurs when the Moon temporarily blocks the light from the Sun. Within the narrow, 60- to 70-mile-wide band stretching from Oregon to South Carolina called the path of totality, the Moon completely blocked out the Sun’s face; elsewhere in North America, the Moon covered only a part of the star, leaving a crescent-shaped Sun visible in the sky.

During this exciting event, we were collecting your images and reactions online. 

Here are a few images of this celestial event…take a look:

This composite image, made from 4 frames, shows the International Space Station, with a crew of six onboard, as it transits the Sun at roughly five miles per second during a partial solar eclipse from, Northern Cascades National Park in Washington. Onboard as part of Expedition 52 are: NASA astronauts Peggy Whitson, Jack Fischer, and Randy Bresnik; Russian cosmonauts Fyodor Yurchikhin and Sergey Ryazanskiy; and ESA (European Space Agency) astronaut Paolo Nespoli.

Credit: NASA/Bill Ingalls

The Bailey’s Beads effect is seen as the moon makes its final move over the sun during the total solar eclipse on Monday, August 21, 2017 above Madras, Oregon.

Credit: NASA/Aubrey Gemignani

This image from one of our Twitter followers shows the eclipse through tree leaves as crescent shaped shadows from Seattle, WA.

Credit: Logan Johnson

“The eclipse in the palm of my hand”. The eclipse is seen here through an indirect method, known as a pinhole projector, by one of our followers on social media from Arlington, TX.

Credit: Mark Schnyder

Through the lens on a pair of solar filter glasses, a social media follower captures the partial eclipse from Norridgewock, ME.

Credit: Mikayla Chase

While most of us watched the eclipse from Earth, six humans had the opportunity to view the event from 250 miles above on the International Space Station. European Space Agency (ESA) astronaut Paolo Nespoli captured this image of the Moon’s shadow crossing America.

Credit: Paolo Nespoli

This composite image shows the progression of a partial solar eclipse over Ross Lake, in Northern Cascades National Park, Washington. The beautiful series of the partially eclipsed sun shows the full spectrum of the event. 

Credit: NASA/Bill Ingalls

In this video captured at 1,500 frames per second with a high-speed camera, the International Space Station, with a crew of six onboard, is seen in silhouette as it transits the sun at roughly five miles per second during a partial solar eclipse, Monday, Aug. 21, 2017 near Banner, Wyoming.

Credit: NASA/Joel Kowsky

To see more images from our NASA photographers, visit: https://www.flickr.com/photos/nasahqphoto/albums/72157685363271303

Make sure to follow us on Tumblr for your regular dose of space: http://nasa.tumblr.com

People think coding / debugging means highly concentrated furious typing, but mostly it’s just angrily staring at the screen for long periods of time waiting for the problem to solve itself