Thoughts on making Movie Barcode Quiz (Angular.js, Parse, Web Scraper Chrome)

About a week ago my friend Garrett showed me this wonderful art tumblr called Movie Barcode.  In it, an anonymous curator posts anamorphic stills generated by stitching together single-pixel-wide bars captured from the film frames.  I spent a few hours putting together The Movie Barcode Quiz.  If you want to look at the code, here is the github project.

I wanted to share a few details about the tools I used to build this project.

Getting the Data

I’m a data hoarder, and have been building web scrapers for about as long as I’ve been programming.  I think writing a scraping script is a good way to dive into a new programming language because it involves a healthy amount of library and paradigm coverage.  For web scraping, you’ll need to figure out how to fetch the URL, traverse the data to get what you want, model and sanitize it, then store it.

In the past, scraping has been painful.

I love data, but I do not like writing data scrapers.  The process is painful, time consuming, and requires a lot of maintenance.  Ain’t nobody got time.

There are a lot of frameworks in a lot of languages which, depending on your needs, may ameliorate some of the pain of building and maintaining a scraper.   At PyCon 2014 I got to see a demo of Portia, which is an intelligent GUI built atop scrapy.   However, like many other projects in the Python world, scrapy and Portia feel heavy handed for what they actually accomplish.  I still use them and have a dedicated vagrant sandbox for projects requiring more granular scraping tools, but the overhead involved in setting up a new scrapers has dissuaded me from starting many new data projects.  I wanted to try something lighter.

Web Scraper (Chrome plugin)

It makes sense to me that your web scraper live in the browser along with the data, and I’ve wanted to try one of the browser plugins I’ve been hearing about.  The impeccably named Chrome plugin, Web Scraper is a perfect tool for getting data out of the browser and into a CSV file.  For my purposes I had an excellently annotated list of film titles + years, along with links to their movie barcode images on the movie barcode index page.  Using Web Scraper’s hierarchy tool, it was intuitive to describe to the crawler that it should open each of the links and save the image src.  After about an hour and a half I had a nice big CSV with over 16k film titles and image urls.

Parse as a backend

Since the end result is going to be a (literally) Single Page App running on Angular, I don’t need a complex back end.  In fact, the only data I’m transmitting are image URLs and movie titles.  Theoretically, I could just put all of this data in a static file and download it with the rest of the assets, and there are some cases where that might make sense.

For this app I’m using Parse as my back end, which has good and bad features.  Among the good for this project are its high availability, its APIs, and its price (free-ish).

And among the bad: First, Parse has metered requests, so you actually have to slow JavaScript down as you’re seeding the database.  This isn’t a big deal, and it gave me the rare opportunity to implement my own craptastic while loop.  I realize there are more elegant ways to do this.

Randomness with Parse

Second, Parse has a limited query language.  It’s built on MongoDB but only exposes a subset of Mongo’s capabilities.  The most glaring shortcoming as far as this app is concerned is the complete lack of a “find random” function.  Getting random rows is basically all I need this backend to do.  Oy.

A way around this is to set a monotonically incrementing index for each of the rows, and then to generate a random number on the client side to send with the request for the films.  Works fine.

The Web App (Angular.js)

I’ve been living and working almost exclusively in JavaScript land for the past several months and I have to say it’s really nice here.  The tooling and package management via NPM, Grunt, Bower, etc., give you superpowers and the ability to stay focused on what you’re trying to build rather than burning thought power on how you’re going to build it.

To set up this project I used the Yeoman angular generator, and its awesome bower + grunt integration for live reloading and building.

Trip to Uruguay

My friend James (Diego) and I had to write a story for our Spanish class in high school about an imaginary trip to a South American country.  I think this is why Señora hated us.


Jeeves has infiltrated my Internet browser. He is rather insistent I take a trip somewhere. I called Diego to see if he could come in and exterminate Jeeves somehow. In the midst of Diego’s endeavor, he booked us on a flight to Montevideo. We leave tomorrow at 7.

Day One

This is not Uruguay. This is not a Spanish speaking land in even the vaguest sense.

Diego and I boarded the plane this morning. We ate our in-flight peanuts. Diego locked himself in the bathroom for no apparent reason. After the flight attendants broke the door down, he calmly returned to his chair next to me and said he had seen the ground through the toilet drain. I requested another seat.

Out flight took twenty minutes. We got to the hotel, and Diego screamed bloody murder. He ran to the room and locked himself in that bathroom. He later told me that Vikings fans make him nauseous. While he was weeping, I called the airport. I explained that when I boarded the plane this morning, I was under the impression that I would be in Montevideo, Uruguay. Instead, I was in Montevideo, Minnesota. If it were my intention to end up in Minnesota, I would have driven.

I was impressed by the courtesy of the airport staff, who booked Diego and I on a fight, half-price, even, to leave that afternoon. I pounded on the bathroom door and relayed the news rather loudly to Diego. He said he’d think about it.


Day Two

We arrived in Montevideo, Uruguay early this morning. Diego was asleep, and I can’t say I blame him. He was completely exhausted from his three and a half hour struggle to free himself from the duct tape I had strapped him to his seat with. He ceased, finally, when I explained to him how necessary the duct tape was. If there’s one thing Diego takes seriously, it’s chemical warfare.

After Checking into our Sheraton Hotel, Diego and I explored the many facets of Montevideo. We spent a great deal of the day in the “Ciudad Vieja,” or the “Old Town.” I was captivated by the 18th century architecture which was apparent most places you looked. Diego said he thought it’d be older, while excusing himself to use the latrine. I laughed when he realized it was an antique.

Next, we visited the Mercado del Puerto, where Diego bought himself some fruit souvenirs. I bought a marionette puppet from a meager merchant who called himself No Toque. I must have given him enough Pesos to begin with, because he didn’t even barter with me. I decided I would name my Puppet after him.

When Diego, No Toque, and I got back to the hotel, we were hot like skillets. We decided a swim in the pool was in order. Diego is a fine swimmer. He made me watch him do cannon balls and other water tricks he had perfected on his own. I sink when I swim, and decided it was safest to remain in the shallow end about where the steps were. No Toque was partial to the hot tub. He fell to pieces before I got him back to the room, but not before he ate Diego’s souvenirs.


Day Three

Today we travel by bus to Punta del Este where we will visit the “Isla de los Lobos,” which is home to one of the largest sea lion colonies in the world. Diego couldn’t wait to get going, and had everything packed and ready this morning before I even opened my eyes. When I asked him why he was so excited, he told me he has always wanted to shake the Big Hand in the Sand.

The bus ride from Montevideo to Punta de la Esta was fairly uneventful. A tour guide pointed out all the areas of interest, especially those places Bess, our bus driver, could not go. Bess growled at all of us as we were exiting the bus.

Diego ran straight for the Hand in the Sand. He tossed me his camera so that I could take a picture of him hugging the massive index finger. He spent an hour searching for the giant pair of fingernail clippers that belonged to the hand. But to no avail, apparently. I am not sure, as I left him to go work on my tan.

The beach was pretty busy for a weekday. I suppose if we had a beach in Wayland, I’d be skipping school, too. Diego and I went snorkeling. I filled his snorkel with sand. He was spitting for hours.

As the sun was setting, and the beaches were closing down, Diego told me there was one last thing he had to do. He ran to the shore through a flock of standing birds who quickly scattered at the sight of Diego. They flew noisily in all directions, and resettled again in different places. Diego returned at a slower pace with a shell in his hand. “So I can hear the ocean,” he said, as I caught a glimpse of a hermit crab peeking out of the opening. I didn’t say anything, though. Instead I just smiled.

We waited at the bus stop for Bess to come pick us up. Luckily, it was a different driver whose only vice appeared to be a tendency to bite his nails. We got back to the hotel, and went right to sleep. We have a long day ahead of us.


Day Four

This morning we checked out of the hotel. Diego took one too many complementary mints, and received an evil stare from the receptionist that said, “never come back”. Diego sent him back a look that explained, “I don’t intend to,” while grabbing another fist-full of mints. The receptionist, greatly offended by this act of war, placed the bowl of mints under the counter. Diego shot back a resentful gaze. Concerned for our safety, I broke the duel of stares by thumping a suitcase against Diego’s chest. “Carry this,” I told him, and lead the way to the exit.

When we got outside, we noticed it had begun raining. We were under the main entrance car tent, so we kept dry enough. I asked Diego how we were going to get to our next destination, which was the Lago Artificial, Uruguay’s largest lake. No sooner had I asked that when a car pulled up right in front of us under the tent. An American man clad in a business coat stepped out and said half to us, half to himself, “Oh, I hope I’m not late.”

Diego answered, “No, you’re not late. Checkout isn’t till 11.”

The business man handed Diego his car keys and hurried inside. As he was passing through the doors, he shouted back at us, “Be good to her!” I guess he meant his car, as I wasn’t wearing my kilt today.

Diego and I loaded our luggage into the trunk of the car. We were eager to get on our way- it was a long drive to Lago Artificial. The car was incredibly roomy. Diego estimated it to be at least $120,000, and why anyone would just give it to us was beyond him.

The early morning drizzle became a downpour and we turned up the radio to drown out the hypnotic rubbing of the windshield wipers. Diego began to really get into the Spanish pop music. On some songs, he would blink his hazard lights and honk along. Cars pulled off the road to let him and his bad self through. He’d honk politely at them as we passed.

We reached Lago Artificial in the afternoon and looked into the waters. Diego said the lake looked pretty real to him. It was still raining, so there wasn’t much else to do except sit in the car and play checkers. I beat Diego three times, so he owes me a pop once Lent is over.

We stayed the night at a small Inn on the edge of the lake.


Day Five

On our last day, Diego wanted to stop at a town he found on the internet called Trenta y Tres. Since there is an airport there, I didn’t argue. We didn’t have anything else in mind other than to get there, get to the airport, and get home. Just the thrill of seeing the water tower was enough to make Diego giggle like a schoolgirl. He snapped a few pictures as we pulled into the airport parking lot.

On our way through customs, the x-ray of Diego’s bags revealed a small, crab-shaped rock. The inspectors made Diego leave the wildlife on the Uruguayan soil. This included the shell he picked up from the beach. He boarded the plane with no souvenir except a pen he stole from the security guard and a bag full of mints. I told him it wasn’t a wasted trip. At least he had his memory. And his pictures. And a luxury vehicle that would be shipped to us within the week.


“They’re made out of meat.”


“Meat. They’re made out of meat.”


“There’s no doubt about it. We picked up several from different parts of the planet, took them aboard our recon vessels, and probed them all the way through. They’re completely meat.”

“That’s impossible. What about the radio signals? The messages to the stars?”

“They use the radio waves to talk, but the signals don’t come from them. The signals come from machines.”

“So who made the machines? That’s who we want to contact.”

They made the machines. That’s what I’m trying to tell you. Meat made the machines.”

“That’s ridiculous. How can meat make a machine? You’re asking me to believe in sentient meat.”

“I’m not asking you, I’m telling you. These creatures are the only sentient race in that sector and they’re made out of meat.”

“Maybe they’re like the orfolei. You know, a carbon-based intelligence that goes through a meat stage.”

“Nope. They’re born meat and they die meat. We studied them for several of their life spans, which didn’t take long. Do you have any idea what’s the life span of meat?”

“Spare me. Okay, maybe they’re only part meat. You know, like the weddilei. A meat head with an electron plasma brain inside.”

“Nope. We thought of that, since they do have meat heads, like the weddilei. But I told you, we probed them. They’re meat all the way through.”

“No brain?”

“Oh, there’s a brain all right. It’s just that the brain is made out of meat! That’s what I’ve been trying to tell you.”

“So … what does the thinking?”

“You’re not understanding, are you? You’re refusing to deal with what I’m telling you. The brain does the thinking. The meat.”

“Thinking meat! You’re asking me to believe in thinking meat!”

“Yes, thinking meat! Conscious meat! Loving meat. Dreaming meat. The meat is the whole deal!  Are you beginning to get the picture or do I have to start all over?”

“Omigod. You’re serious then. They’re made out of meat.”

“Thank you. Finally. Yes. They are indeed made out of meat. And they’ve been trying to get in touch with us for almost a hundred of their years.”

“Omigod. So what does this meat have in mind?”

“First it wants to talk to us. Then I imagine it wants to explore the Universe, contact other sentiences, swap ideas and information. The usual.”

“We’re supposed to talk to meat.”

“That’s the idea. That’s the message they’re sending out by radio. ‘Hello. Anyone out there. Anybody home.’ That sort of thing.”

“They actually do talk, then. They use words, ideas, concepts?”
“Oh, yes. Except they do it with meat.”

“I thought you just told me they used radio.”

“They do, but what do you think is on the radio? Meat sounds. You know how when you slap or flap meat, it makes a noise? They talk by flapping their meat at each other. They can even sing by squirting air through their meat.”

“Omigod. Singing meat. This is altogether too much. So what do you advise?”

“Officially or unofficially?”


“Officially, we are required to contact, welcome and log in any and all sentient races or multibeings in this quadrant of the Universe, without prejudice, fear or favor. Unofficially, I advise that we erase the records and forget the whole thing.”

“I was hoping you would say that.”

“It seems harsh, but there is a limit. Do we really want to make contact with meat?”

“I agree one hundred percent. What’s there to say? ‘Hello, meat. How’s it going?’ But will this work? How many planets are we dealing with here?”

“Just one. They can travel to other planets in special meat containers, but they can’t live on them. And being meat, they can only travel through C space. Which limits them to the speed of light and makes the possibility of their ever making contact pretty slim. Infinitesimal, in fact.”

“So we just pretend there’s no one home in the Universe.”

“That’s it.”

“Cruel. But you said it yourself, who wants to meet meat? And the ones who have been aboard our vessels, the ones you probed? You’re sure they won’t remember?”

“They’ll be considered crackpots if they do. We went into their heads and smoothed out their meat so that we’re just a dream to them.”

“A dream to meat! How strangely appropriate, that we should be meat’s dream.”

“And we marked the entire sector unoccupied.”

“Good. Agreed, officially and unofficially. Case closed. Any others? Anyone interesting on that side of the galaxy?”

“Yes, a rather shy but sweet hydrogen core cluster intelligence in a class nine star in G445 zone. Was in contact two galactic rotations ago, wants to be friendly again.”

“They always come around.”

“And why not? Imagine how unbearably, how unutterably cold the Universe would be if one were all alone …”

By Terry Bisson

The Egg

The Egg

This is a story by Andy Weir.  Please go buy his very good books.

You were on your way home when you died.

It was a car accident. Nothing particularly remarkable, but fatal nonetheless. You left behind a wife and two children. It was a painless death. The EMTs tried their best to save you, but to no avail. Your body was so utterly shattered you were better off, trust me.

And that’s when you met me.

“What… what happened?” You asked. “Where am I?”

“You died,” I said, matter-of-factly. No point in mincing words.

“There was a… a truck and it was skidding…”

“Yup,” I said.

“I… I died?”

“Yup. But don’t feel bad about it. Everyone dies,” I said.

You looked around. There was nothingness. Just you and me. “What is this place?” You asked. “Is this the afterlife?”

“More or less,” I said.

“Are you god?” You asked.

“Yup,” I replied. “I’m God.”

“My kids… my wife,” you said.

“What about them?”

“Will they be all right?”

“That’s what I like to see,” I said. “You just died and your main concern is for your family. That’s good stuff right there.”

You looked at me with fascination. To you, I didn’t look like God. I just looked like some man. Or possibly a woman. Some vague authority figure, maybe. More of a grammar school teacher than the almighty.

“Don’t worry,” I said. “They’ll be fine. Your kids will remember you as perfect in every way. They didn’t have time to grow contempt for you. Your wife will cry on the outside, but will be secretly relieved. To be fair, your marriage was falling apart. If it’s any consolation, she’ll feel very guilty for feeling relieved.”

“Oh,” you said. “So what happens now? Do I go to heaven or hell or something?”

“Neither,” I said. “You’ll be reincarnated.”

“Ah,” you said. “So the Hindus were right,”

“All religions are right in their own way,” I said. “Walk with me.”

You followed along as we strode through the void. “Where are we going?”

“Nowhere in particular,” I said. “It’s just nice to walk while we talk.”

“So what’s the point, then?” You asked. “When I get reborn, I’ll just be a blank slate, right? A baby. So all my experiences and everything I did in this life won’t matter.”

“Not so!” I said. “You have within you all the knowledge and experiences of all your past lives. You just don’t remember them right now.”

I stopped walking and took you by the shoulders. “Your soul is more magnificent, beautiful, and gigantic than you can possibly imagine. A human mind can only contain a tiny fraction of what you are. It’s like sticking your finger in a glass of water to see if it’s hot or cold. You put a tiny part of yourself into the vessel, and when you bring it back out, you’ve gained all the experiences it had.

“You’ve been in a human for the last 48 years, so you haven’t stretched out yet and felt the rest of your immense consciousness. If we hung out here for long enough, you’d start remembering everything. But there’s no point to doing that between each life.”

“How many times have I been reincarnated, then?”

“Oh lots. Lots and lots. An in to lots of different lives.” I said. “This time around, you’ll be a Chinese peasant girl in 540 AD.”

“Wait, what?” You stammered. “You’re sending me back in time?”

“Well, I guess technically. Time, as you know it, only exists in your universe. Things are different where I come from.”

“Where you come from?” You said.

“Oh sure,” I explained “I come from somewhere. Somewhere else. And there are others like me. I know you’ll want to know what it’s like there, but honestly you wouldn’t understand.”

“Oh,” you said, a little let down. “But wait. If I get reincarnated to other places in time, I could have interacted with myself at some point.”

“Sure. Happens all the time. And with both lives only aware of their own lifespan you don’t even know it’s happening.”

“So what’s the point of it all?”

“Seriously?” I asked. “Seriously? You’re asking me for the meaning of life? Isn’t that a little stereotypical?”

“Well it’s a reasonable question,” you persisted.

I looked you in the eye. “The meaning of life, the reason I made this whole universe, is for you to mature.”

“You mean mankind? You want us to mature?”

“No, just you. I made this whole universe for you. With each new life you grow and mature and become a larger and greater intellect.”

“Just me? What about everyone else?”

“There is no one else,” I said. “In this universe, there’s just you and me.”

You stared blankly at me. “But all the people on earth…”

“All you. Different incarnations of you.”

“Wait. I’m everyone!?”

“Now you’re getting it,” I said, with a congratulatory slap on the back.

“I’m every human being who ever lived?”

“Or who will ever live, yes.”

“I’m Abraham Lincoln?”

“And you’re John Wilkes Booth, too,” I added.

“I’m Hitler?” You said, appalled.

“And you’re the millions he killed.”

“I’m Jesus?”

“And you’re everyone who followed him.”

You fell silent.

“Every time you victimized someone,” I said, “you were victimizing yourself. Every act of kindness you’ve done, you’ve done to yourself. Every happy and sad moment ever experienced by any human was, or will be, experienced by you.”

You thought for a long time.

“Why?” You asked me. “Why do all this?”

“Because someday, you will become like me. Because that’s what you are. You’re one of my kind. You’re my child.”

“Whoa,” you said, incredulous. “You mean I’m a god?”

“No. Not yet. You’re a fetus. You’re still growing. Once you’ve lived every human life throughout all time, you will have grown enough to be born.”

“So the whole universe,” you said, “it’s just…”

“An egg.” I answered. “Now it’s time for you to move on to your next life.”

And I sent you on your way.


Selections from a fourth grader’s journal

What I want my teacher to know about me
I enjoy doing magic shows and pupet shows. I make lots of home videos. I’m also a ventrilaqest (well actually I use my dummy for stunts in my movies). When I growup I’d like to be a few things, a Magician an actor, a teacher, and maybe a stuntman.

Teacher’s comment, in purple gel ink: Donald, I hope you will use your many talents for future book presentations!

I am Donald Whalen. I am shorter than most of the boys in my class, and taller than most of the girls in my class. Many people think I’m from a diffrent planet. Their right. I’m from the planet Bearth. Whoaaa! Did I say Bearth? I meant Earth, the most diffrent planet in the soler system.

I think I will write about an invention I’ve been thinking about.

Problem: When people don’t know a road or are sleepy They usually miss their turns.

Teacher’s comment: Yes- and then what?

I get lonely lots of times. Only my mom and I understand the feeling. What’s weird about it is I can be in a crowded room, and have no reason to get the feeling or in an empty room. It kind of feels like a nerve snapping. You feel that sudden jolt, then every thing seems far away from you.

I did feel realy lonly when we got back from church on Easter and my cousins had already left.

The conclusion of a fourth grader’s research paper:
“Sea turtles have come and evolved from the huge creature that they were to the middleing creature that tehy are. Who knows what they’ll look like in the many centurys to come. We may never know if we don’t start protecting, and saving them.”