Google shows off AI tool for reading handwritten text by rewriting it digitally
AI’s handwriting is now better than mine
When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.
GoogleResearch is showing off a new way to use AI to read handwriting that might radically change how machines convert what you put on paper into digital letters. The InkSight system transforms photos of handwritten words into digital text by leveraging AI without the need for any devices as intermediaries.
The idea is to replace the sometimes fallible optical character recognition (OCR) with AI that can emulate how humans actually learn to read, specifically by rewriting existing text to learn what whole words look like and mean. Doing so required the researchers to tutor the AI in both recognizing and mimicking handwriting by humans.
“Digital note-taking is gaining popularity, offering a durable, editable, and easily indexable way of storing notes in the vectorized form, known as digital ink. However, a substantial gap remains between this way of note-taking and traditional pen-and-paper note-taking, a practice still favored by a vast majority,” the researchers explain in theirpaper. “Our approach combines reading and writing priors, allowing training a model in the absence of large amounts of paired samples, which are difficult to obtain. To our knowledge, this is the first work that effectively derenders handwritten text in arbitrary photos with diverse visual characteristics and backgrounds.”
InkSight is more than just an alternative technique. It makes for more accurate results in circumstances that aren’t ideal. For instance, if the photo is taken in dim light, has partially obscured text, or is on a confusing background when examined with OCR. The researchers found that humans could read 87% of the InkSight-made tracings of text. Two-thirds were good enough that people couldn’t tell them from actual handwriting; you can see below how it looks when InkSight works.
Penned by AI
If you like writing things by hand, InkSight has some potential benefits. Imagine writing by hand in a paper notebook, then showing the notes to your camera to instantly make them searchable and organize them in context with previous notes on physical pages. If you’re like me and have particularly messy handwriting, InkSight could help turn your chicken scratch into typewritten text that is still accurate to what you scribble.
On a bigger scale, this could be a crucial tool for deciphering and converting handwritten text from across the centuries into digital form. Even when the text is in a language without much of a digital presence, InkSight could help preserve handwriting to help build up training sources for those languages.
Google isn’t the only place where AI tools to decipher handwriting are underway. For example,Amazon’snew Kindle Scribeupgrades the e-reader’s ability to transform handwritten notes into legible text. There’s alsoGoodnotes, a digital notetaking app that can read handwriting, and recently debutedhandwriting editingtools using its Goodnotes Smart Ink technology to turn handwriting into typed text. The added tools let you edit handwritten notes as if they were typed, including aligning notes, copying and pasting, and reflowing the text to make it more logical.
Get the best Black Friday deals direct to your inbox, plus news, reviews, and more.
Sign up to be the first to know about unmissable Black Friday deals on top tech, plus get all your favorite TechRadar content.
You might also like…
Eric Hal Schwartz is a freelance writer for TechRadar with more than 15 years of experience covering the intersection of the world and technology. For the last five years, he served as head writer for Voicebot.ai and was on the leading edge of reporting on generative AI and large language models. He’s since become an expert on the products of generative AI models, such as OpenAI’s ChatGPT, Anthropic’s Claude, Google Gemini, and every other synthetic media tool. His experience runs the gamut of media, including print, digital, broadcast, and live events. Now, he’s continuing to tell the stories people want and need to hear about the rapidly evolving AI space and its impact on their lives. Eric is based in New York City.
Google Gemini is set to finally reach its full potential – and take over from Google Assistant – thanks to a major upgrade
Anthropic’s new Claude 3.5 Haiku AI model is 4 times more expensive than its predecessor
This super-cheap HP Victus 15 gaming laptop just dropped to its lowest price yet
