New algorithm helps read QR codes on uneven surfaces

Sometimes, we try to capture a QR code with a good digital camera on a smartphone, but the reading eventually fails. This usually happens when the QR code itself is of poor image quality, or if it has been printed on surfaces that are not flat—deformed or with irregularities of unknown pattern—such as the wrapping of a courier package or a tray of prepared food.

Now, a team from the University of Barcelona and the Universitat Oberta de Catalunya has designed a methodology that facilitates the recognition of QR codes in physical environments where reading is more complicated. The paper is published in the journal Pattern Recognition Letters.

The new system does not depend absolutely on the underlying topography, and is applicable to QR codes that can be found on tubular surfaces (bottles), food trays, etc. It is the first technological proposal capable of combining a generalist methodology and two-dimensional barcodes to facilitate the recognition of digital information.

The study’s first author is Professor Ismael Benito from the UB’s Faculty of Physics and Department of Electronic and Biomedical Engineering and the UOC’s Department of Computer Science, Multimedia and Telecommunications Studies. All the authors have participated in different positions in the creation of ColorSensing, SL, a UB spin-off company in the field of smart labeling.

Why are some QR codes difficult to read?

QR codes are a variation of the typical barcode, capable of collecting information in computer language—in a two-dimensional matrix of black and white pixels—when scanned with a scanning device. They facilitate access to data of interest, save time and resources such as paper, and have revolutionized the way users access information in the digital realm.

However, it is sometimes difficult to scan a barcode correctly. According to Benito, from the UB’s Department of Electronic and Biomedical Engineering and former technology director of ColorSensing, this happens, “first of all, because of the quality of the image. Although many people today have access to good digital cameras, they cannot always capture the QR image well.

“Secondly, the print quality of the QR code and the colors used—with good contrast—is sometimes not satisfactory. Finally, if the printing surface is not flat enough and not parallel to the capture plane, it is also difficult to capture the information in the code.”

“For example, all these factors come into play when we try to capture a Bicing QR with the mobile app: the surface is not flat—it is a cylinder—and if we try to capture the QR too close, the deformation of the surface becomes evident and the reading fails—5–10 centimeters; if we move too far away, the QR becomes too small and the capture is not good—1 meter; if we are in an intermediate range, the apparent distortion of the surface is reduced and the quality is suitable for capturing it—30–50 centimeters,” explains Benito.

An algorithm that exploits properties of QR codes

The study, which is part of Ismael Benito’s doctoral thesis at the UB, presents a new algorithm that takes advantage of the QR’s own characteristics—i.e., the code’s internal patterns—to extract the underlying surface on which the code is positioned.

The texture of this surface is recovered by a generalist adjustment based on mathematical functions known as splines, which allow the topography of the surface to be adjusted locally. Benito points out that “they are functions that adapt locally to the ups and downs of the surface, and form a technique that was originally widely used in fields such as geology or photographic editing to adjust or generate deformations in surfaces.”

There are still many technological challenges to improving the whole process of QR code recognition.

In the case of commercial applications activated by the user’s code reader, the expert explains that “the main challenge is to be able to provide correct and reliable readings. We are also working hard to ensure that the codes cannot be attacked by modification techniques, for example, with a fake URL that can capture data with small modifications to the code.”

“In the case of industry, where captures are done in controlled environments, the main challenge is to reduce the speed of capture,” says Benito.