CA2429507A1 - Writing guide for a free-form document editor - Google Patents

Writing guide for a free-form document editor Download PDF

Info

Publication number
CA2429507A1
CA2429507A1 CA002429507A CA2429507A CA2429507A1 CA 2429507 A1 CA2429507 A1 CA 2429507A1 CA 002429507 A CA002429507 A CA 002429507A CA 2429507 A CA2429507 A CA 2429507A CA 2429507 A1 CA2429507 A1 CA 2429507A1
Authority
CA
Canada
Prior art keywords
ink
handwriting
detected
writing
guide
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CA002429507A
Other languages
French (fr)
Other versions
CA2429507C (en
Inventor
Alex J. Simmons
Kentaro Urata
Peter Loren Engrav
Christopher H. Pratley
Owen C. Braun
Stuart J. Stuple
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Corp
Original Assignee
Microsoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corp filed Critical Microsoft Corp
Publication of CA2429507A1 publication Critical patent/CA2429507A1/en
Application granted granted Critical
Publication of CA2429507C publication Critical patent/CA2429507C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/166Editing, e.g. inserting or deleting
    • G06F40/171Editing, e.g. inserting or deleting by use of digital ink
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/18Extraction of features or characteristics of the image
    • G06V30/18086Extraction of features or characteristics of the image by performing operations within image blocks or by using histograms
    • G06V30/18095Summing image-intensity values; Projection and histogram analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/32Digital ink
    • G06V30/36Matching; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition

Abstract

Displaying a writing guide for a free-form document editor by detecting electronic ink, classifying the detected ink as corresponding to handwriting or a drawing, and displaying a handwriting guide or a drawing guide in response to classifying the detected ink as handwriting or a drawing, respectively. The writing guides can provide feedback regarding the classification of the detected ink and can be resized as additional ink is detected. The handwriting guide can comprise a writing area, writing guide portion, bullet guide, indent marker, character notches, gesture guide, or sacrificial guide to create meaningful structure within an outline object of an electronic document.

Claims (27)

1. A computer-implemented method for displaying a writing guide for a free-form document editor, comprising the steps of detecting electronic ink comprising an ink stroke;
classifying the detected ink as corresponding to one of handwriting and a drawing; and displaying a handwriting writing guide in response to classifying the detected ink as handwriting.
2. The method according to Claim 1, wherein said classifying step comprises the steps of:
setting a threshold aspect ratio indicating whether the detected ink comprises handwriting or a drawing;
determining a length and height of the detected ink;
computing an ink aspect ratio for the detected ink based on the determined length and height;
determining whether the ink aspect ratio is greater than the threshold aspect ratio; and classifying the detected ink as handwriting in response to a determination that the ink aspect ratio is greater than the threshold aspect ratio.
3. The method according to Claim 2, further comprising the steps of:
setting a minimum stroke threshold corresponding to a predetermined number of ink strokes to detect in said detecting step before classifying the detected ink as handwriting or a drawing;
determining a detected number of ink strokes for the detected ink; and determining whether the detected number of ink strokes is greater than the minimum stroke threshold, wherein said classifying step is performed in response to a determination that the detected number of ink strokes is greater than the minimum stroke threshold.
4. The method according to Claim 3, wherein the minimum stroke threshold comprises a predetermined number of ink strokes for classifying the detected ink as handwriting and a different predetermined number of ink strokes for classifying the detected ink as a drawing.
5. The method according to Claim 1, wherein said handwriting writing guide comprises a writing area into which a user inputs the electronic ink.
6. The method according to Claim 5, wherein said displaying step comprises the step of determining dimensions of the writing area based on a bounding rectangle that encompasses the detected ink.
7. The method according to Claim 6, wherein said step of determining dimensions comprises the step of determining a right, left, top, and baseline dimension of the writing area based on the bounding rectangle that encompasses the detected ink.
8. The method according to Claim 7, wherein said step of determining dimensions further comprises the step of increasing the writing area by adding predetermined spacings to one of the top and baseline dimensions.
9. The method according to Claim 6, wherein said step of determining dimensions comprises the steps of:
creating a histogram representing a distribution of the detected ink on a line;
and calculating the baseline dimension of the writing area based on the histogram.
10. The method according to Claim 9, wherein said creating step comprises the steps of:
dividing the line into a plurality of segments;
measuring an amount of the detected ink in each segment; and creating the histogram based on the amount of detected ink in each segment.
11. The method according to Claim 10, wherein said calculating step comprises the steps of:
setting an ink removal threshold corresponding to an amount of ink below which a segment is removed from the writing area;
determining whether the amount of detected ink in a segment is less than the ink removal threshold; and removing the segment from the writing area to establish the baseline dimension of the writing area in response to a determination that the amount of detected ink in the segment is less that the ink removal threshold.
12. The method according to Claim 6, wherein said step of determining dimensions comprises the steps of:
converting a portion of the detected ink to text; and calculating a baseline dimension of the writing area based on the ink-to-text conversion of said converting step.
13. The method according to Claim 6, further comprising the steps of:
setting a resizing threshold corresponding to an amount of ink outside the writing area for resizing the dimensions of the writing area;
detecting an amount of ink outside the writing area;
determining whether the amount of ink outside the writing area is greater than the resizing threshold; and resizing the writing area dimensions in response to a determination that the amount of ink outside the writing area is greater than the resizing threshold.
14. The method according to Claim 6, further comprising the steps of:
setting a resizing threshold corresponding to an amount of ink outside the writing area for resizing the dimensions of the writing area;
establishing a plurality of segments inside and outside the writing area;
detecting an amount of ink in each segment;
determining whether the amount of ink in segments outside the writing area is greater than the resizing threshold; and increasing the writing area dimensions in response to a determination that the amount of ink outside the writing area is greater than the resizing threshold.
15. A computer-implemented method for displaying a writing guide for a free-form document editor, comprising the steps of:
detecting electronic ink;
classifying the detected ink as corresponding to one of handwriting and a drawing; and displaying a handwriting writing guide comprising a writing area into which a user inputs the electronic ink, in response to classifying the detected ink as handwriting.
16. The method according to Claim 15, wherein said displaying step comprises displaying the handwriting writing guide further comprising a next level indent indicator dividing the writing area into a first level region and a second level region, the first and second level regions corresponding to different levels of a structured outline.
17. The method according to Claim 16, wherein said displaying step comprises displaying the handwriting writing guide further comprising a bullet region corresponding to bullet outline of the structured outline.
18. The method according to Claim 17, further comprising the step of activating the bullet region to display the bullet outline.
19. The method according to Claim 15, wherein said displaying step comprises displaying the handwriting writing guide further comprising character notches that divide the writing area into character boxes for individual characters of a language.
20. The method according to Claim 15, wherein said handwriting writing guide comprises a gesture handwriting writing guide.
21. The method according to Claim 15, wherein said handwriting writing guide comprises a sacrificial handwriting writing guide.
22. The method according to Claim 15, wherein said handwriting writing guide comprises a nested heading handwriting writing guide.
23. A computer-implemented method for displaying a writing guide for a free-form document editor, comprising the steps of:
detecting electronic ink;
classifying the detected ink as corresponding to one of handwriting and a drawing; and displaying a drawing guide in response to classifying the detected ink as a drawing.
24. The method according to Claim 23, wherein said displaying step comprises the steps of:
determining dimensions of an original bounding rectangle that encompasses the detected ink; and establishing dimensions of the drawing guide based on the original bounding rectangle dimensions.
25. The method according to Claim 24, further comprising the steps of:
detecting ink outside the original bounding rectangle;
determining dimensions of a new bounding rectangle that encompasses the ink detected outside the original bounding rectangle; and reestablishing the dimensions of the drawing guide based on the new bounding rectangle dimensions.
26. The method according to Claim 24, wherein said displaying step further comprises determining dimensions of a hot zone around the original bounding rectangle, and wherein said establishing step comprises establishing the dimensions of the drawing guide based on the bounding rectangle and hot zone dimensions.
27. The method according to Claim 24, further comprising the step of displaying a handwriting writing guide in response to classifying the detected ink as handwriting.
CA2429507A 2002-06-28 2003-05-26 Writing guide for a free-form document editor Expired - Fee Related CA2429507C (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/186,388 2002-06-28
US10/186,388 US7062090B2 (en) 2002-06-28 2002-06-28 Writing guide for a free-form document editor

Publications (2)

Publication Number Publication Date
CA2429507A1 true CA2429507A1 (en) 2003-12-28
CA2429507C CA2429507C (en) 2011-03-29

Family

ID=29718024

Family Applications (1)

Application Number Title Priority Date Filing Date
CA2429507A Expired - Fee Related CA2429507C (en) 2002-06-28 2003-05-26 Writing guide for a free-form document editor

Country Status (4)

Country Link
US (2) US7062090B2 (en)
EP (2) EP1376390B1 (en)
JP (1) JP4486325B2 (en)
CA (1) CA2429507C (en)

Families Citing this family (64)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7916124B1 (en) 2001-06-20 2011-03-29 Leapfrog Enterprises, Inc. Interactive apparatus using print media
US7079713B2 (en) 2002-06-28 2006-07-18 Microsoft Corporation Method and system for displaying and linking ink objects with recognized text and objects
US7751623B1 (en) 2002-06-28 2010-07-06 Microsoft Corporation Writing guide for a free-form document editor
US20040117724A1 (en) * 2002-12-03 2004-06-17 Microsoft Corporation Extensible schema for ink
US7721226B2 (en) 2004-02-18 2010-05-18 Microsoft Corporation Glom widget
US7831933B2 (en) 2004-03-17 2010-11-09 Leapfrog Enterprises, Inc. Method and system for implementing a user interface for a device employing written graphical elements
US7853193B2 (en) 2004-03-17 2010-12-14 Leapfrog Enterprises, Inc. Method and device for audibly instructing a user to interact with a function
US7659890B2 (en) 2004-03-19 2010-02-09 Microsoft Corporation Automatic height adjustment for electronic highlighter pens and mousing devices
US20060005151A1 (en) * 2004-07-02 2006-01-05 Adobe Systems Graphical interface for adjustment of text selections
US7551779B2 (en) 2005-03-17 2009-06-23 Microsoft Corporation Word or character boundary-based scratch-out gesture recognition
US20060267966A1 (en) * 2005-05-24 2006-11-30 Microsoft Corporation Hover widgets: using the tracking state to extend capabilities of pen-operated devices
US7680332B2 (en) * 2005-05-30 2010-03-16 Microsoft Corporation Grouping lines in freeform handwritten text
US7961943B1 (en) * 2005-06-02 2011-06-14 Zeevi Eli I Integrated document editor
US7922099B1 (en) 2005-07-29 2011-04-12 Leapfrog Enterprises, Inc. System and method for associating content with an image bearing surface
US7936339B2 (en) * 2005-11-01 2011-05-03 Leapfrog Enterprises, Inc. Method and system for invoking computer functionality by interaction with dynamically generated interface regions of a writing surface
US7774722B2 (en) * 2006-01-31 2010-08-10 Microsoft Corporation Creation and manipulation of canvases based on ink strokes
US8599143B1 (en) 2006-02-06 2013-12-03 Leapfrog Enterprises, Inc. Switch configuration for detecting writing pressure in a writing device
JP4759743B2 (en) * 2006-06-06 2011-08-31 国立大学法人 東京大学 Object display processing device, object display processing method, and object display processing program
JP4752066B2 (en) * 2006-07-07 2011-08-17 国立大学法人 東京大学 Handwriting input processing device, handwriting input processing method, and program for handwriting input processing
US8261967B1 (en) 2006-07-19 2012-09-11 Leapfrog Enterprises, Inc. Techniques for interactively coupling electronic content with printed media
US20080168478A1 (en) 2007-01-07 2008-07-10 Andrew Platzer Application Programming Interfaces for Scrolling
US7844915B2 (en) 2007-01-07 2010-11-30 Apple Inc. Application programming interfaces for scrolling operations
US20080168402A1 (en) 2007-01-07 2008-07-10 Christopher Blumenberg Application Programming Interfaces for Gesture Operations
US8407589B2 (en) * 2007-04-20 2013-03-26 Microsoft Corporation Grouping writing regions of digital ink
US7945097B2 (en) * 2007-05-21 2011-05-17 Microsoft Corporation Classifying digital ink into a writing or a drawing
US8645827B2 (en) 2008-03-04 2014-02-04 Apple Inc. Touch event model
US8717305B2 (en) 2008-03-04 2014-05-06 Apple Inc. Touch event model for web pages
US8416196B2 (en) 2008-03-04 2013-04-09 Apple Inc. Touch event model programming interface
JP5169369B2 (en) * 2008-03-25 2013-03-27 富士ゼロックス株式会社 Handwriting information processing apparatus and program
KR101477808B1 (en) * 2008-08-08 2014-12-30 엘지전자 주식회사 Method and apparatus for searching a manual in image display device
US9311112B2 (en) 2009-03-16 2016-04-12 Apple Inc. Event recognition
US8285499B2 (en) 2009-03-16 2012-10-09 Apple Inc. Event recognition
US8566045B2 (en) 2009-03-16 2013-10-22 Apple Inc. Event recognition
US9684521B2 (en) 2010-01-26 2017-06-20 Apple Inc. Systems having discrete and continuous gesture recognizers
US8473842B2 (en) 2010-05-12 2013-06-25 Microsoft Corporation Contour based flow layout
US8218875B2 (en) 2010-06-12 2012-07-10 Hussein Khalid Al-Omari Method and system for preprocessing an image for optical character recognition
US8552999B2 (en) 2010-06-14 2013-10-08 Apple Inc. Control selection approximation
US9223471B2 (en) 2010-12-28 2015-12-29 Microsoft Technology Licensing, Llc Touch screen control
US9298363B2 (en) * 2011-04-11 2016-03-29 Apple Inc. Region activation for touch sensitive surface
JP5683411B2 (en) * 2011-08-25 2015-03-11 シャープ株式会社 Display device and display program
JP5418576B2 (en) * 2011-11-28 2014-02-19 コニカミノルタ株式会社 Information browsing apparatus and display control program
US9007390B2 (en) 2012-05-15 2015-04-14 Evernote Corporation Creation and manipulation of hand drawn objects with automatic grouping
KR20140030372A (en) * 2012-08-27 2014-03-12 삼성전자주식회사 Method for processing customized page and mobile device
US9589538B2 (en) 2012-10-17 2017-03-07 Perceptive Pixel, Inc. Controlling virtual objects
KR102091000B1 (en) * 2013-05-31 2020-04-14 삼성전자 주식회사 Method and apparatus for processing data using user gesture
US9733716B2 (en) 2013-06-09 2017-08-15 Apple Inc. Proxy gesture recognizer
US9696810B2 (en) 2013-06-11 2017-07-04 Microsoft Technology Licensing, Llc Managing ink content in structured formats
KR102135947B1 (en) * 2013-08-26 2020-07-21 삼성전자주식회사 Method for resizing window area and electronic device for the same
CN104423825A (en) * 2013-09-02 2015-03-18 联想(北京)有限公司 Electronic equipment and information processing method thereof
JP6109020B2 (en) * 2013-09-10 2017-04-05 インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation Document dividing / combining method, apparatus, and program.
JP6279373B2 (en) * 2014-03-20 2018-02-14 株式会社東芝 Correction apparatus, method and program
US9552473B2 (en) 2014-05-14 2017-01-24 Microsoft Technology Licensing, Llc Claiming data from a virtual whiteboard
US10270819B2 (en) 2014-05-14 2019-04-23 Microsoft Technology Licensing, Llc System and method providing collaborative interaction
US9341374B2 (en) 2014-06-03 2016-05-17 Siemens Energy, Inc. Fuel nozzle assembly with removable components
JP6392036B2 (en) * 2014-09-03 2018-09-19 株式会社東芝 Electronic apparatus and method
US20160154555A1 (en) * 2014-12-02 2016-06-02 Lenovo (Singapore) Pte. Ltd. Initiating application and performing function based on input
US9613263B2 (en) * 2015-02-27 2017-04-04 Lenovo (Singapore) Pte. Ltd. Ink stroke grouping based on stroke attributes
US10416877B2 (en) * 2015-08-25 2019-09-17 Myscript System and method of guiding handwriting input
US10324618B1 (en) * 2016-01-05 2019-06-18 Quirklogic, Inc. System and method for formatting and manipulating digital ink
US10755029B1 (en) 2016-01-05 2020-08-25 Quirklogic, Inc. Evaluating and formatting handwritten input in a cell of a virtual canvas
US10691316B2 (en) 2016-03-29 2020-06-23 Microsoft Technology Licensing, Llc Guide objects for drawing in user interfaces
US10163004B2 (en) * 2017-03-30 2018-12-25 Konica Minolta Laboratory U.S.A., Inc. Inferring stroke information from an image
US20190155895A1 (en) * 2017-11-20 2019-05-23 Google Llc Electronic text pen systems and methods
US11372518B2 (en) * 2020-06-03 2022-06-28 Capital One Services, Llc Systems and methods for augmented or mixed reality writing

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4377803A (en) * 1980-07-02 1983-03-22 International Business Machines Corporation Algorithm for the segmentation of printed fixed pitch documents
US4741045A (en) * 1983-09-23 1988-04-26 Dest Corporation Optical character isolation system, apparatus and method
JPH0650538B2 (en) * 1984-03-28 1994-06-29 株式会社日立製作所 How to specify the recognition mode for handwriting / figure recognition
US5321770A (en) * 1991-11-19 1994-06-14 Xerox Corporation Method for determining boundaries of words in text
US5583542A (en) * 1992-05-26 1996-12-10 Apple Computer, Incorporated Method for deleting objects on a computer display
US5517578A (en) * 1993-05-20 1996-05-14 Aha! Software Corporation Method and apparatus for grouping and manipulating electronic representations of handwriting, printing and drawings
US5500937A (en) * 1993-09-08 1996-03-19 Apple Computer, Inc. Method and apparatus for editing an inked object while simultaneously displaying its recognized object
EP0677817B1 (en) * 1994-04-15 2000-11-08 Canon Kabushiki Kaisha Page segmentation and character recognition system
US5687254A (en) * 1994-06-06 1997-11-11 Xerox Corporation Searching and Matching unrecognized handwriting
DE69428527T2 (en) * 1994-07-04 2002-05-08 Hewlett Packard Co Kritzlervergleich
JP2944439B2 (en) * 1994-12-27 1999-09-06 シャープ株式会社 Handwritten character input device and method
JPH09319504A (en) * 1996-05-31 1997-12-12 Sanyo Electric Co Ltd Document processor, document processing method and storage medium
US6054990A (en) * 1996-07-05 2000-04-25 Tran; Bao Q. Computer system with handwriting annotation
JP4098880B2 (en) * 1997-06-06 2008-06-11 松下電器産業株式会社 Information retrieval device
US6952803B1 (en) * 1998-12-29 2005-10-04 Xerox Corporation Method and system for transcribing and editing using a structured freeform editor
US6384815B1 (en) * 1999-02-24 2002-05-07 Hewlett-Packard Company Automatic highlighting tool for document composing and editing software
US6625583B1 (en) * 1999-10-06 2003-09-23 Goldman, Sachs & Co. Handheld trading system interface
WO2001054054A1 (en) * 2000-01-19 2001-07-26 California Institute Of Technology Word recognition using silhouette bar codes
US20030215145A1 (en) * 2002-05-14 2003-11-20 Microsoft Corporation Classification analysis of freeform digital ink input

Also Published As

Publication number Publication date
JP2004062891A (en) 2004-02-26
US20060257025A1 (en) 2006-11-16
EP1376390A3 (en) 2007-02-28
CA2429507C (en) 2011-03-29
JP4486325B2 (en) 2010-06-23
US7062090B2 (en) 2006-06-13
US20040001627A1 (en) 2004-01-01
EP1376390A2 (en) 2004-01-02
EP2869212A1 (en) 2015-05-06
EP1376390B1 (en) 2014-12-03
US7409089B2 (en) 2008-08-05

Similar Documents

Publication Publication Date Title
CA2429507A1 (en) Writing guide for a free-form document editor
US5517578A (en) Method and apparatus for grouping and manipulating electronic representations of handwriting, printing and drawings
JP5729930B2 (en) Method for distinguishing text lines in a document
KR100658119B1 (en) Apparatus and Method for Recognizing Character
Pal et al. Multioriented and curved text lines extraction from Indian documents
US20170337423A1 (en) Method for Structural Analysis and Recongnigiton of Handwritten Mathematical Formula in Natural Scene Image
Pal et al. Identification of different script lines from multi-script documents
US8452097B2 (en) Apparatus and method for extracting circumscribed rectangles of characters in transplantable electronic document
US20140184610A1 (en) Shaping device and shaping method
JP7244223B2 (en) Identifying emphasized text in electronic documents
Roy et al. Morphology based handwritten line segmentation using foreground and background information
JP2008176521A (en) Pattern separation extraction program, pattern separation extraction apparatus and pattern separation extraction method
JP2015049618A (en) Handwritten document processing device, handwritten document processing method, and handwritten document processing program
US8768061B2 (en) Post optical character recognition determination of font size
CN103761220B (en) A kind of text composition technical method of information software
CN103455816B (en) Stroke width extraction method and device and character recognition method and system
Liang et al. Performance evaluation of document layout analysis algorithms on the UW data set
JP5950700B2 (en) Image processing apparatus, image processing method, and program
CN112926582B (en) Text detection method based on adaptive feature selection and scale loss function
CN111738252A (en) Method and device for detecting text lines in image and computer system
CN101901333B (en) Method for segmenting word in text image and identification device using same
CN103106309A (en) Sketch behavior identification method and sketch behavior identification system during computer aided design process
CN115223172A (en) Text extraction method, device and equipment
Tsai et al. Efficiently extracting and classifying objects for analyzing color documents
CN102567302B (en) Method and device for identifying typesetting form

Legal Events

Date Code Title Description
EEER Examination request
MKLA Lapsed

Effective date: 20190527