Home > Articles

This chapter is from the book

8.2 The Compiler API

In the preceding sections, you saw how to interact with code in a scripting language. Now we turn to a different scenario: Java programs that compile Java code. There are quite a few tools that need to invoke the Java compiler, such as:

  • Development environments

  • Java teaching and tutoring programs

  • Build and test automation tools

  • Templating tools that process snippets of Java code, such as JavaServer Pages (JSP)

In the past, applications invoked the Java compiler by calling undocumented classes in the jdk/lib/tools.jar library. Nowadays, a public API for compilation is a part of the Java platform, and it is no longer necessary to use tools.jar. This section explains the compiler API.

8.2.1 Compiling the Easy Way

It is very easy to invoke the compiler. Here is a sample call:

JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
OutputStream outStream = . . .;
OutputStream errStream = . . .;
int result = compiler.run(null, outStream, errStream, "-sourcepath", "src", "Test.java");

A result value of 0 indicates successful compilation.

The compiler sends output and error messages to the provided streams. You can set these parameters to null, in which case System.out and System.err are used. The first parameter of the run method is an input stream. As the compiler takes no console input, you can always leave it as null. (The run method is inherited from a generic Tool interface, which allows for tools that read input.)

The remaining parameters of the run method are simply the arguments that you would pass to javac if you invoked it on the command line. These can be options or file names.

8.2.2 Using Compilation Tasks

You can have even more control over the compilation process with a CompilationTask object. In particular, you can

  • Control the source of program code—for example, by providing code in a string builder instead of a file

  • Control the placement of class files—for example, by storing them in a database

  • Listen to error and warning messages as they occur during compilation

  • Run the compiler in the background

The location of source and class files is controlled by a JavaFileManager. It is responsible for determining JavaFileObject instances for source and class files. A JavaFileObject can correspond to a disk file, or it can provide another mechanism for reading and writing its contents.

To listen to error messages, install a DiagnosticListener. The listener receives a Diagnostic object whenever the compiler reports a warning or error message. The DiagnosticCollector class implements this interface. It simply collects all diagnostics so that you can iterate through them after the compilation is complete.

A Diagnostic object contains information about the problem location (including file name, line number, and column number) as well as a human-readable description.

To obtain a CompilationTask object, call the getTask method of the JavaCompiler class. You need to specify:

  • A Writer for any compiler output that is not reported as a Diagnostic, or null to use System.err

  • A JavaFileManager, or null to use the compiler’s standard file manager

  • A DiagnosticListener

  • Option strings, or null for no options

  • Class names for annotation processing, or null if none are specified (we’ll discuss annotation processing later in this chapter)

  • JavaFileObject instances for source files

You need to provide the last three arguments as Iterable objects. For example, a sequence of options might be specified as

Iterable<String> options = Arrays.asList("-g", "-d", "classes");

Alternatively, you can use any collection class.

If you want the compiler to read source files from disk, you can ask the StandardJavaFileManager to translate the file name strings or File objects to JavaFileObject instances. For example,

StandardJavaFileManager fileManager = compiler.getStandardFileManager(null, null, null);
Iterable<JavaFileObject> fileObjects = fileManager.getJavaFileObjectsFromStrings(fileNames);

However, if you want the compiler to read source code from somewhere other than a disk file, you need to supply your own JavaFileObject subclass. Listing 8.3 shows the code for a source file object with data contained in a StringBuilder. The class extends the SimpleJavaFileObject convenience class and overrides the getCharContent method to return the content of the string builder. We’ll use this class in our example program in which we dynamically produce the code for a Java class and then compile it.

The CompilationTask interface extends the Callable<Boolean> interface. You can pass it to an Executor for execution in another thread, or you can simply invoke the call method. A return value of Boolean.FALSE indicates failure.

Callable<Boolean> task = new JavaCompiler.CompilationTask(null, fileManager, diagnostics,
      options, null, fileObjects);
if (!task.call())
   System.out.println("Compilation failed");

If you simply want the compiler to produce class files on disk, you need not customize the JavaFileManager. However, our sample application will generate class files in byte arrays and later read them from memory, using a special class loader. Listing 8.4 defines a class that implements the JavaFileObject interface. Its openOutputStream method returns the ByteArrayOutputStream into which the compiler will deposit the bytecodes.

It turns out a bit tricky to tell the compiler’s file manager to use these file objects. The library doesn’t supply a class that implements the StandardJavaFileManager interface. Instead, you subclass the ForwardingJavaFileManager class that delegates all calls to a given file manager. In our situation, we only want to change the getJavaFileForOutput method. We achieve this with the following outline:

JavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
fileManager = new ForwardingJavaFileManager<JavaFileManager>(fileManager)
   {
      public JavaFileObject getJavaFileForOutput(Location location, final String className,
            Kind kind, FileObject sibling) throws IOException
      {
         return custom file object
      }
   };

In summary, call the run method of the JavaCompiler task if you simply want to invoke the compiler in the usual way, reading and writing disk files. You can capture the output and error messages, but you need to parse them yourself.

If you want more control over file handling or error reporting, use the CompilationTask interface instead. Its API is quite complex, but you can control every aspect of the compilation process.

Listing 8.3 compiler/StringBuilderJavaSource.java

 1   package compiler;
 2
 3   import java.net.*;
 4   import javax.tools.*;
 5
 6   /**
 7    * A Java source that holds the code in a string builder.
 8    * @version 1.00 2007-11-02
 9    * @author Cay Horstmann
10    */
11   public class StringBuilderJavaSource extends SimpleJavaFileObject
12   {
13      private StringBuilder code;
14
15      /**
16       * Constructs a new StringBuilderJavaSource.
17       * @param name the name of the source file represented by this file object
18       */
19      public StringBuilderJavaSource(String name)
20      {
21         super(URI.create("string:///" + name.replace('.', '/') + Kind.SOURCE.extension),
22               Kind.SOURCE);
23         code = new StringBuilder();
24      }
25
26      public CharSequence getCharContent(boolean ignoreEncodingErrors)
27      {
28         return code;
29      }
30
31      public void append(String str)
32      {
33         code.append(str);
34         code.append('\n');
35      }
36   }

Listing 8.4 compiler/ByteArrayJavaClass.java

 1   package compiler;
 2
 3   import java.io.*;
 4   import java.net.*;
 5   import javax.tools.*;
 6
 7   /**
 8    * A Java class that holds the bytecodes in a byte array.
 9    * @version 1.00 2007-11-02
10    * @author Cay Horstmann
11    */
12   public class ByteArrayJavaClass extends SimpleJavaFileObject
13   {
14      private ByteArrayOutputStream stream;
15
16      /**
17       * Constructs a new ByteArrayJavaClass.
18       * @param name the name of the class file represented by this file object
19       */
20      public ByteArrayJavaClass(String name)
21      {
22         super(URI.create("bytes:///" + name), Kind.CLASS);
23         stream = new ByteArrayOutputStream();
24      }
25
26      public OutputStream openOutputStream() throws IOException
27      {
28         return stream;
29      }
30
31      public byte[] getBytes()
32      {
33         return stream.toByteArray();
34      }
35   }

8.2.3 An Example: Dynamic Java Code Generation

In the JSP technology for dynamic web pages, you can mix HTML with snippets of Java code, for example

<p>The current date and time is <b><%= new java.util.Date() %></b>.</p>

The JSP engine dynamically compiles the Java code into a servlet. In our sample application, we use a simpler example and generate dynamic Swing code instead. The idea is that you use a GUI builder to lay out the components in a frame and specify the behavior of the components in an external file. Listing 8.5 shows a very simple example of a frame class, and Listing 8.6 shows the code for the button actions. Note that the constructor of the frame class calls an abstract method addEventHandlers. Our code generator will produce a subclass that implements the addEventHandlers method, adding an action listener for each line in the action.properties file. (We leave it as the proverbial exercise to the reader to extend the code generation to other event types.)

We place the subclass into a package with the name x, which we hope is not used anywhere else in the program. The generated code has the form

package x;
public class Frame extends SuperclassName
{
   protected void addEventHandlers()
   {
      componentName1.addActionListener(new java.awt.event.ActionListener()
        {
           public void actionPerformed(java.awt.event.ActionEvent) { code for event handler1 }
        });
     // repeat for the other event handlers ...
   }
}

The buildSource method in the program of Listing 8.7 builds up this code and places it into a StringBuilderJavaSource object. That object is passed to the Java compiler.

We use a ForwardingJavaFileManager with a getJavaFileForOutput method that constructs a ByteArrayJavaClass object for every class in the x package. These objects capture the class files generated when the x.Frame class is compiled. The method adds each file object to a list before returning it so that we can locate the bytecodes later. Note that compiling the x.Frame class produces a class file for the main class and one class file per listener class.

After compilation, we build a map that associates class names with bytecode arrays. A simple class loader (shown in Listing 8.8) loads the classes stored in this map.

We ask the class loader to load the class that we just compiled. Then, we construct and display the application’s frame class.

ClassLoader loader = new MapClassLoader(byteCodeMap);
Class<?> cl = loader.loadClass("x.Frame");
Frame frame = (JFrame) cl.newInstance();
frame.setVisible(true);

When you click the buttons, the background color changes in the usual way. To see that the actions are dynamically compiled, change one of the lines in action.properties, for example, like this:

yellowButton=panel.setBackground(java.awt.Color.YELLOW); yellowButton.setEnabled(false);

Run the program again. Now the Yellow button is disabled after you click it. Also, have a look at the code directories. You will not find any source or class files for the classes in the x package. This example demonstrates how you can use dynamic compilation with in-memory source and class files.

Listing 8.5 buttons2/ButtonFrame.java

 1   package buttons2;
 2   import javax.swing.*;
 3
 4   /**
 5    * A frame with a button panel.
 6    * @version 1.00 2007-11-02
 7    * @author Cay Horstmann
 8    */
 9   public abstract class ButtonFrame extends JFrame
10   {
11      public static final int DEFAULT_WIDTH = 300;
12      public static final int DEFAULT_HEIGHT = 200;
13
14      protected JPanel panel;
15      protected JButton yellowButton;
16      protected JButton blueButton;
17      protected JButton redButton;
18
19      protected abstract void addEventHandlers();
20
21      public ButtonFrame()
22      {
23         setSize(DEFAULT_WIDTH, DEFAULT_HEIGHT);
24
25         panel = new JPanel();
26         add(panel);
27
28         yellowButton = new JButton("Yellow");
29         blueButton = new JButton("Blue");
30         redButton = new JButton("Red");
31
32         panel.add(yellowButton);
33         panel.add(blueButton);
34         panel.add(redButton);
35
36         addEventHandlers();
37      }
38   }

Listing 8.6 buttons2/action.properties

1 yellowButton=panel.setBackground(java.awt.Color.YELLOW);
2 blueButton=panel.setBackground(java.awt.Color.BLUE);

Listing 8.7 compiler/CompilerTest.java

 1   package compiler;
 2
 3   import java.awt.*;
 4   import java.io.*;
 5   import java.util.*;
 6   import java.util.List;
 7   import javax.swing.*;
 8   import javax.tools.*;
 9   import javax.tools.JavaFileObject.*;
10
11   /**
12    * @version 1.01 2016-05-10
13    * @author Cay Horstmann
14    */
15   public class CompilerTest
16   {
17      public static void main(final String[] args) throws IOException, ClassNotFoundException
18      {
19         JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
20
21         final List<ByteArrayJavaClass> classFileObjects = new ArrayList<>();
22
23         DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<>();
24
25         JavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
26         fileManager = new ForwardingJavaFileManager<JavaFileManager>(fileManager)
27            {
28               public JavaFileObject getJavaFileForOutput(Location location, final String className,
29                     Kind kind, FileObject sibling) throws IOException
30               {
31                  if (className.startsWith("x."))
32                  {
33                     ByteArrayJavaClass fileObject = new ByteArrayJavaClass(className);
34                     classFileObjects.add(fileObject);
35                     return fileObject;
36                  }
37                  else return super.getJavaFileForOutput(location, className, kind, sibling);
38               }
39            };
40
41         String frameClassName = args.length == 0 ? "buttons2.ButtonFrame" : args[0];
42         JavaFileObject source = buildSource(frameClassName);
43         JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager,
44               null, Arrays.asList(source));
45         Boolean result = task.call();
46
47         for (Diagnostic<? extends JavaFileObject> d : diagnostics.getDiagnostics())
48            System.out.println(d.getKind() + ": " + d.getMessage(null));
49         fileManager.close();
50         if (!result)
51         {
52            System.out.println("Compilation failed.");
53            System.exit(1);
54         }
55
56         EventQueue.invokeLater(() ->
57            {
58               try
59               {
60                  Map<String, byte[]> byteCodeMap = new HashMap<>();
61                  for (ByteArrayJavaClass cl : classFileObjects)
62                     byteCodeMap.put(cl.getName().substring(1), cl.getBytes());
63                  ClassLoader loader = new MapClassLoader(byteCodeMap);
64                  JFrame frame = (JFrame) loader.loadClass("x.Frame").newInstance();
65                  frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
66                  frame.setTitle("CompilerTest");
67                  frame.setVisible(true);
68               }
69               catch (Exception ex)
70               {
71                  ex.printStackTrace();
72               }
73            });
74      }
75
76      /*
77       * Builds the source for the subclass that implements the addEventHandlers method.
78       * @return a file object containing the source in a string builder
79       */
80      static JavaFileObject buildSource(String superclassName)
81        throws IOException, ClassNotFoundException
82      {
83        StringBuilderJavaSource source = new StringBuilderJavaSource("x.Frame");
84        source.append("package x;\n");
85        source.append("public class Frame extends " + superclassName + " {");
86        source.append("protected void addEventHandlers() {");
87        final Properties props = new Properties();

88        props.load(Class.forName(superclassName).getResourceAsStream("action.properties"));
89        for (Map.Entry<Object, Object> e : props.entrySet())
90        {
91           String beanName = (String) e.getKey();
92           String eventCode = (String) e.getValue();
93           source.append(beanName + ".addActionListener(event -> {");
94           source.append(eventCode);
95           source.append("});");
96        }
97        source.append("} }");
98        return source;
99     }
100  }

Listing 8.8 compiler/MapClassLoader.java

 1   package compiler;
 2
 3   import java.util.*;
 4
 5   /**
 6    * A class loader that loads classes from a map whose keys are class names and whose values are
 7    * byte code arrays.
 8    * @version 1.00 2007-11-02
 9    * @author Cay Horstmann
10    */
11   public class MapClassLoader extends ClassLoader
12   {
13      private Map<String, byte[]> classes;
14
15      public MapClassLoader(Map<String, byte[]> classes)
16      {
17         this.classes = classes;
18      }
19
20      protected Class<?> findClass(String name) throws ClassNotFoundException
21      {
22         byte[] classBytes = classes.get(name);
23         if (classBytes == null) throw new ClassNotFoundException(name);
24         Class<?> cl = defineClass(name, classBytes, 0, classBytes.length);
25         if (cl == null) throw new ClassNotFoundException(name);
26         return cl;
27      }
28   }

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020