Tuesday, 18 October 2016

We are living in the future

The lady I clean for is blind, and she has just got a new app for her iPad - Tap Tap See. I've downloaded it too, to play with :-)

It's an object recognition program. She tried another one before, which was fun but a bit tricky to use. This one, however, is easier and also gives really useful information, because it reads the text, if any, in the photo.

For me it's just fun, but for a blind person it has a practical purpose. She was given a gift set for her birthday, and with a bit of trial and error she discovered it was Crabtree & Evelyn toiletries, what fragrance they were, and what was in the different bottles.

Now that's useful!

Here's my results:
 [edit: it appears there is also some human input in the results]
"turned off computer monitor"
(Useful to know it's turned off.)
"empty 2 slice bread toaster"
(Actually it's 4 slices, but two slots, so I'll forgive it.)
"surgical spirit b.p labeled plastic bottle"
(Exactly the info you'd want to know)
"mod podge matte-mat-mate 236 ml"
(Glad I'm not american - why didn't it read the fluid ounces?)
"black android smartphone turned off"
"canada printed black ceramic mug"
Glad it didn't mention the untidy dining table :-)

"irn bru sugar free plastic bottle"
Now some posters from above my desk:
"family fun day poster"
"sithrah poster"
"white green and black the burrell collection"
(OK, I'd have preferred if it had described the picture)
"does this show the real face of jesus text"
"black and yellow painting mounted on wall"
"male and female body chart"
"people's job print on white printer paper illustration"
"turned on flatscreen computer monitor"
(I am ashamed at the state of my desk. But not ashamed enough to do anything about it.)
I had taken some photos before for the previous app (on my old phone, hence blurry photos) - here are the results with this app:
"brown wooden wall frame"
"red pink and white rooster artwork"
"brown and white football shaped textile"
(Actually it's an african shield, but amazingly like an american football!)
"white lamp shade"
"white and green bird plastic wall decor"
(Yes, I have these on my wall - I like them!)
"bird's can on grey blue green and yellow floral textile"
(Good reading! Not sure why it's so interested in the tablecloth.)
"white toilet paper in gray steel toilet paper holder"
"white light switch mounted on wall turned on"
(It's a socket, not a light switch. I wonder what country the app is from?)
"green obtuse plant on the brown plastic pot"
(Obtuse? And the pot is made of bark.)
"girl in pink and red lace ruffle dress doll"
"yellow car park near house"
"green plant near bare tree"
"brown concrete animal figurine"
(Actually stone. Interesting it knows it's an animal, but not that it's a snake. My dad carved this in college)
"brown and black dog plush toy on teal white tool box"
(Actually a file, but partly hidden, so pretty good. And it knew he wasn't a real dog.)
"brown bird figurine"
"buttered bread with spread"
(Scrambled egg on toast!)
"green grapes on white green and blue floral round plate"
(It's good at extrapolating.)
"pink and white flower on white ceramic flower vase"
"person in plaid skirts and sling bag"
(I'm sure my dad would be thrilled with that description of his kilt and sporran!!!)


Neil said...

Apparently it uses a combination of computer image recognition and real humans - that explains how it does so well.

I know computer vision is progressing a lot but I didn't think it was that good yet.

Kirsty said...

Ah, that explains it!