Building Parsers With Java

About

Features



Description

• Copyright 2001
• Dimensions: 7-3/8" x 9-1/4"
• Pages: 408
• Edition: 1st

• Book
• ISBN-10: 0-201-71962-2
• ISBN-13: 978-0-201-71962-8

No programming language can solve every problem, but Java can be extended to solve a far wider range of problems through the use of parsers -- "mini-languages" that bridge the gap between humans and computers, and offer targeted solutions for specific problem domains. In Building Parsers with Java, Steven John Metsker presents the first complete, start-to-finish guide to building parsers with Java. Metsker first explains what a parser is, introduces the building blocks of applied parsers, shows how to compose new parsers from existing ones; and walks step-by-step through designing, coding, and testing a working parser. Next, he presents in-depth coverage of creating Java-based parsers for a wide range of problem domains. You'll learn how to create parsers that can read elements of a data language; how to transform a grammar, ensuring the correct behavior of operators in a language; how to build arithmetic and regular expression parsers, and more. Metsker shows how to extend existing parser toolkits, and introduces a logic engine that can be used in a wide range of parsers, including both logic and query languages.



Downloads

Source Code

Download the Source Code for this book.



Extras

Related Article

Regular Expressions Do Not Solve All Problems

Author's Site

Click below for Web Resources related to this title:
Author's Web Site



Sample Content

Downloadable Sample Chapter

Click below for Sample Chapter related to this title:
metskch1.pdf

metskch2.pdf

metskch3.pdf

Table of Contents

(NOTE: Each chapter concludes with a Summary.)

Foreword.


Preface.

Who Should Read This Book.

Using the Toolkit Code and the Sample Code.

Contents of the CD.

Applying the Code on the CD.

Hello World.

Coding Style.

Related Books.

Theoretical Context.

Yacc and Lex and Bison and Flex.

About the Cover.

Acknowledgments.

1. Introduction.

The Role of Parsers.

What Is a Language?

The Organization of This Book.

2. The Elements of a Parser.

What Is a Parser?

Parser Collaborations.

Assemblies.

The Assembly Class Interfaces.

Token and Character Assemblies.

Tokenizing.

Default and Custom Tokenization.

Assembly Appearance.

Assembly Summary.

The Parser Hierarchy.

The Composition of a Parser.

Terminal Parsers.

Using Terminals.

Word Terminals.

Num Terminals.

Literals.

Caseless Literals.

Symbols.

Quoted Strings.

Composite Parsers.

Repetition.

Alternation and Sequence.

Composing a Parser.

The Empty Parser.

Parser Summary.

Assemblers.

Parsers Use Assemblers.

Assemblers Work On Assemblies.

Elements Above.

3. Building a Parser.

Design Overview.

Deciding to Tokenize.

Designing Assemblers.

The Collaboration of Parsers, Assemblers, and Assemblies.

Using an Assembly's Stack.

Assemblers Plug In to Parser Composites.

A Language to Plug In To: Minimath.

Calculating a Minimath Result.

The Minimath Parser as an Object.

Building a Target.

Making a Target Cloneable.

Grammars: A Shorthand for Parsers.

Standard Grammar Shorthand.

Top-Down Grammar Design.

Example: Designing a Grammar for a Track Robot.

A Track Robot Grammar.

Checking for Left Recursion and Cycles.

Translating a Grammar to Code.

Translate Quoted Strings.

Translate Sequences.

Translate Alternations.

Translate Terminals.

Create a Subparser for Each Rule.

Option 1: Declare Each Subparser.

Option 2: Arrange Subparsers as Methods.

Add a Start Method.

Completing a Parser.

Control Pushing.

Design the Target.

Plug In Assemblers.

4. Testing a Parser.

Feature Testing.

Random Testing.

Ambiguity Testing.

Terminal Ambiguity.

Special Tokenizers and Targets.

5. Parsing Data Languages.

The Role of Data Languages.

A Data Language Example.

A Coffee Grammar.

A Tokenizing Problem.

Coffee Assemblers.

Coffee Assembler Code.

Translating the Coffee Grammar to Code.

Data Language Parser Summary.

Parsers with XML.

A Brief History of XML.

The Evolution of XML.

An XML Example.

Helpers.

ShowCoffeeXML.

6. Transforming a Grammar.

The Role of Grammar Transformation.

Ensuring Correct Associativity.

Eliminating Left Recursion.

An Algorithm.

Ensuring Proper Precedence.

Eliminating Parser Class Loops.

7. Parsing Arithmetic.

Building an Arithmetic Parser.

Conventional Symbols.

Conventional Precedence.

Conventional Associativity.

An Arithmetic Grammar.

Arithmetic Assemblers.

Assembler Code.

An Arithmetic Grammar Parser.

8. Parsing Regular Expressions.

The Role of Regular Expressions.

Building a Regular Expression Parser.

A Regular Expression Grammar.

Regular Expression Assemblers.

Assembler Code.

A Regular Expression Parser.

9. Advanced Tokenizing.

The Role of a Tokenizer.

Acquiring a Tokenizer.

Tokenizers in Standard Java.

A Token Class.

A Tokenizer Class.

Tokenizer Lookup Tables.

Tokenizer States.

QuoteState.

NumberState.

SlashState.

SymbolState.

WhitespaceState.

WordState.

Setting a Tokenizer's Source.

Customizing a Tokenizer.

The TokenStringSource Utility.

Customizing a State.

Changing Which State the Tokenizer Enters.

Adding a State.

Token Strings.

10. Matching Mechanics.

Introduction.

Parser Matching.

Repetition Matching.

Collection Parsers.

Sequence Matching.

Alternation Matching.

Empty Matching.

Terminal Matching.

Token Terminals.

Character Terminals.

Terminals Summary.

Parser Matching Utilities.

11. Extending the Parser Toolkit.

The Role of New Types of Parsers.

New Terminals.

New Token Types.

New Parser Features.

An Error-Handling Parser.

Tracks in Action.

12. Engines.

Engines versus Interpreters.

The Role of Engines.

Building Blocks.

Structures.

Variables.

Unification.

Facts.

Programs and Queries.

Proofs.

Backtracking.

Rules.

Additional Features of the Engine.

Comparisons.

Arithmetic.

Evaluation.

Not.

Anonymous Variables.

Lists.

13. Logic Programming.

The Role of Logic Languages.

Building Blocks.

Structures.

Variables.

A Logikus Interactive Development Environment.

Unification.

Comparisons.

Rules, Axioms, and Programs.

Proofs.

Variable Scope.

Variable Joins.

Backtracking.

Looping and Halting.

Gateways.

Additional Features of Logikus.

Comments.

Evaluations.

Negation.

Not Dangerous.

Anonymous Variables.

Lists.

Dot Notation.

List Applications.

Member.

Prefix.

Suffix.

Permutation.

Modeling Transitive and Symmetric Relations.

Symmetric Relations.

Example Applications.

An Authorization Program.

Epidemic.

Generate and Test.

Generate and Test in Java.

Altitude Bands.

14. Parsing a Logic Language.

Building a Logic Language Environment.

A Logikus Grammar.

Comments in Logikus.

Logikus Programs.

Logikus Assemblers.

The Logikus Interactive Development Environment.

A Facade for Logikus.

15. Parsing a Query Language.

The Role of Query Languages.

A Sample Database.

Facts, Objects, and Rows.

Jaql.

Jaql Syntax.

Jaql Joins.

Jaql Expressions.

Building a Query Language Environment.

Translating User Queries to Engine Queries.

Natural Joins.

Where Clauses.

Projection.

A Query Builder.

A Speller.

Jaql Grammar.

Creating the Jaql Parser.

Jaql Assemblers.

The Jaql User Environment.

Exception Handling.

Tracks.

Handling Abject Failure.

16. Parsing an Imperative Language.

The Role of Imperative Languages.

Sling.

Sling Programming.

A Basic Sling.

Adding Slings.

Plotting Time.

Line Effects.

Adding Lines.

Cartesian Plots.

Cartesians as Points.

Polar Plots.

For Loops.

Sliders.

A Composite Example.

More Plots.

The Elements of Sling>

Building the Sling Environment.

Building Commands.

A Command Example.

AssignmentCommand.

CommandSequence.

ForCommand.

IfCommand.

NullCommand.

PrintlnCommand.

ReadCommand.

WhileCommand.

Commands Summary.

Sling Commands.

AddFunctionCommand.

AssignFunctionCommand.

Building Runtime Functions.

Function Wrapping.

Umbrella Types.

Runtime Functions in Sling.

Execution Phases.

Prototyping.

Function Evaluation.

Sling Functions.

SlingFunction.

Abs, Ceil, Cos, Floor, Sin, and Tan.

Arithmetic.

Cartesian.

Point.

Polar.

Random.

Scale.

Sling.

Slider.

T.

Variable.

Sling Target.

A Sling Grammar.

Sling Assemblers.

AssignmentAssembler.

ForAssembler.

FunctionAssembler.

NegativeAssembler.

NumAssembler.

PiAssembler.

PlotAssembler.

ScaleAssembler.

SliderAssembler.

VariableAssembler.

A Sling Parser.

Reserved Words.

Assembler Placement.

17. Directions.

Get Started.

Get the Most Out of This Book.

Other Essential Reading.

Keep Going.

Create New Languages for Your Colleagues.

Create New Languages for Children.

Explore Languages as Human Interfaces.

Improve Languages as Receptacles for Human Thought.

Choose Your Own Direction.

Appendix A: UML Twice Distilled.

Classes.

Class Relationships.

Interfaces.

Objects.

References.


Index.

Preface

The premise of this book is that by learning how to work with parsers, you can create new computer languages that exactly fit your domain. When you create a language, you give your language users a new way to control their computers. By learning about parsers, you learn to define the way your users interact with computers using text.

Who Should Read This Book

This book assumes you have a good understanding of Java and would like to learn how to do the following:

• Use a handful of tools to create new computer languages quickly.
• Translate the design of a language into code.
• Create new computer languages with Extensible Markup Language (XML).
• Accept an arithmetic formula from your user and compute its result.
• Accept and apply matching expressions such as th* one.
• Create query languages that fire an engine.
• Program in logic and create a logic language parser.
• Make rules-based programming available to your users from within a Java application.
• Program in Sling, a new computer language that plots the path of a sling.
• Create computer languages that fill niches in the work you do.

Using the Toolkit Code and the Sample Code

This book comes with a CD that contains all the code.

Contents of the CD

The CD includes all the code of the fundamental parser classes, the logic engine, and all the examples. The CD also contains the javadoc documentation from the code, which explains class by class how the code works.

Applying the Code on the CD

The code on the CD is free. It is copyrighted, so you may not claim that you wrote it. Otherwise, you may use the code as you wish.

Hello World

The following program is a sufficient test to verify that you can use the code from the CD. Load the code from the CD into your development environment. Type in the following program, or load it from ShowHello.java on the CD.

package sjm.examples.preface;  import sjm.parse.*; import sjm.parse.tokens.*;   public class ShowHello {   public static void main(String args) {     Terminal t   = new Terminal();     Repetition r = new Repetition(t);          Assembly in  = new TokenAssembly("Hello world!");     Assembly out = r.completeMatch(in);          System.out.println(out.getStack()); } } 

Compiling and running this class prints the following:

Hello, world, !

Once you get this running in your environment, you will be able to use all the fundamental classes and all the examples in this book.

Coding Style

Some features of the coding style in this book may seem unusual. First, this book does not indent method signatures. This practice stems from the fact that the VisualAge development environment exports classes this way, resulting in a pair of curly braces at the end of a class. This convention has the happy effect of allowing a little more space before statements are wrapped within the narrow margins of this book.

Another feature of the coding style in this book that may give you pause is the use of extremely short variable names. Methods in this book nearly always perform a single service and thus are short. Temporary variables are never far from their declarations, and there is usually no need for names longer than one character. For example, it is not difficult in the preceding program to discern that the variable t refers to a Terminal object. In the rare event that two variables of a given type occur in one method, they receive meaningful names, such as in and out in the preceding example.

Comments in the code use javadoc tags such as @param and @exception, but the text usually omits these to save space. Comments for public methods begin with email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information

Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security

Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children

This site is not directed to children under the age of 13.

Marketing

Pearson may send or direct marketing communications to users, provided that

• Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
• Such marketing is consistent with applicable law and Pearson's legal obligations.
• Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
• Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information

If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out

Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information

Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents

California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure

Pearson may disclose personal information, as follows:

• As required by law.
• With the consent of the individual (or their parent, if the individual is a minor)
• In response to a subpoena, court order or legal process, to the extent permitted or required by law
• To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
• In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
• To investigate or address actual or suspected fraud or other illegal activities
• To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
• To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
• To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links

This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact

Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice

We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020