Chapter 8: Online Analysis of Hand-Drawn Strokes for Geometry Learning

This work takes place within the ACTIF project, in the context of the eFran call for projects that aims for active and collaborative learning promotion. This paper presents a pattern recognition and analysis system for Geometry learning in middle school. The goal is to allow students to draw geometric shapes on a pen-based tablet, given a teacher’s instruction. To make the student active, the system have to recognize and analyze on the fly the hand-drawn student’s productions in order to produce real-time visual, corrective, and guidance feedback. We base our work on the visual grammar CD-CMG (Context Driven Constraints Multi-set Grammar), to model the domain prior knowledge and interpret the hand-drawn sketches on the fly. Our first contribution lies in adapting this grammar to the Geometry domain to cover the geometric objects taught in middle school curriculum. Although being expressive enough to model this large scope, the original formalism could not cope with the requirement of real-time analysis, given that the multiple interactions between geometric objects generate combinatorial issues. Our second contribution lies in extending the formalism to obtain acceptable performance for a real-time user interaction system. The first experiments on complex geometric figures drawing scenarios show that the proposed approach allows complexity and interpretation time reduction. We present also our result on another application domain, architecture plan sketching, to prove the generecity of our approach.