Extending Jython

Date: Mar 29, 2002

Article is provided courtesy of New Riders.

Robert W. Bill, author of Jython for Java Programmers, shows Jython programmers tips on extending Jython with Java classes.

Extending Jython means writing Jython modules in Java. Jython modules is used here to mean Java classes that specifically behave like Jython modules. There is a distinction between this and just writing Java classes. Jython can use most any Java class, so there is no requirement to take additional steps to make that Java class look and act like a Jython module. However, there are situations where design begs for a true Jython module. The distinction occurs when a class should allow Jythonisms such as ordered and keyword parameters.

Listing 1 shows a sample Jython module written in Java. You can see that it is nothing more than a Java class. Although the mymod class in listing 1 is very simple, it demonstrates many aspects of creating Jython modules in Java. The parts of interest are the ClassDictInit interface, the related classDictInit method, static modifiers on each method, the __doc__ strings, and the PyException.

Listing 1—A Jython Module Written in Java

// file mymod.java
package org.python.demo.modules;
import org.python.core.*;

public class mymod implements ClassDictInit {

  public static void classDictInit(PyObject dict) {
    dict.__setitem__("__doc__", 
             new PyString("Test class to confirm " +
                   "builtin module"));
    dict.__delitem__("classDictInit");
  }

  public static PyString __doc__fibTest =
    new PyString("fibTest(iteration) "+
           "-> integer");

  public static int fibTest(PyObject[] args, String[] kw) {
    ArgParser ap = new ArgParser("fibTest", args, kw, "iteration");
    int iteration = ap.getInt(0);
    if (iteration < 1)
      throw new PyException(Py.ValueError, 
                 new PyString("Only integers >=1 
allowed"));
    if (iteration == 1 || iteration == 2)
      return iteration;

    return fibTest(new PyObject[] { new PyInteger(iteration-1) }, 
            new String[0]) + 
        fibTest(new PyObject[] { new PyInteger(iteration-2) }, 
            new String[0]);
  }
}

The following is example output from running the mymod class as a module within Jython's interactive shell:

>>> from org.python.demo.modules import mymod
>>> import time
>>>
>>> def test(method, iteration):
...   t1 = time.time()
    results = apply(method, (iteration,))
...   print "The 25th fibonacci iteration is: ", results
...   print "Time elapsed: ", time.time() - t1
...
>>> test(mymod.fibTest, 25)
The 25th fibonacci iteration is: 121393
Time elapsed: 0.8799999952316284

If we continue this example with a comparable fibonacci function written in Jython, it clarifies one advantage to writing modules in Java- performance. You could additionally minimize some of the instantiations in the fibTest method if you passed on implementing Jython's flexible parameter scheme and use just public static int fibTest(int iteration). But even without that tweak, the Java sample above outperforms the Jython version shown here:

>>> def fib(iteration):
...   if iteration < 1: raise ValueError, "Iteration must be >=1"
...   if iteration < 3: return iteration
...   return fib(iteration - 1) + fib(iteration -2)
...
>>> test(fib, 25)
The 25th fibonacci iteration is: 121393
Time elapsed: 1.590000033378601

ClassDictInit

A Jython module written as a Java class may control the module's __dict__ attribute if it implements the ClassDictInit interface. A class that implements the ClassDictInit interface must have a method like the following:

public static void classDictInit(PyObject dict)

Jython calls the classDictInit method when the class is initialized, allowing it to control the attribute names visible within Jython and their implementations. The mymod class in Listing 1 uses the classDictInit to set a __doc__ string and to remove itself, classDictInit, from names visible from Jython. In reality, this is unnecessary because the __doc__ string could be defined as follows:

public static String __doc__="Test class to confirm builtin module";

Without the __doc__ string assignment in classDictInit, the classDictInit method is not needed and would not appear in Jython anyway. The real usefulness of ClassDictInit comes when you have a complicated module implemented in multiple classes or a class includes multiple attributes you need to hide from Jython. All this is done by changing the dictionary object passed to the classDictInit method.

There is an additional means for controlling a Java method's visibility in Jython that is promising because of its simplicity, but is somewhat experimental at the time this was written. This involves the exception org.python.core.PyIgnoreMethodTag. The exception is never actually thrown, but any Java method that declares that it throws this exception is automatically removed from Jython's view.

doc Strings

You can define a module's doc string by assigning a string to the __doc__ key in the dictionary passed to the classDictInit method as shown in Listing 1, or better yet, you can include a static PyString class member named __doc__ in the Java class:

public static __doc__ = new PyString("Some documentation");

The line above defines the module's doc string, but how do you define a function's doc string? The static methods within a Java class become module functions in Jython. In Listing 1, the static fibTest method behaves like the fibTest function within a Jython module. To add a doc string to this function means adding a static PyString called __doc__fibTest. Creating doc strings for other functions means following the same convention of assigning a static PyString to an identifier named __doc__functionName. Listing 1 uses this to assign documentation to the fibTest method than can be retrieved from Jython as mymod.fibTest.__doc__.

Exceptions

To raise a Jython exception requires throwing the PyException class with two arguments: the actual Jython exception, and the exception message (what is referred to as the exception argument). Jython's built-in exceptions are located in the org.python.core.Py class. So, a ValueError is really a Py.ValueError.

raise ValueError, "Invalid Value"

The preceding Jython statement translates to its Java form as follows:

throw PyException(Py.ValueError, "Invalid Value")

Listing 1 uses the PyException to raise a Jython ValueError exception.

Parameters

Jython functions benefit from a rich parameter scheme that includes ordered parameter, keyword parameters, default values, and wild cards. It is possible to emulate this behavior in Java. The following method signature allows a Java method to handle positional and keyword arguments:

public PyObject MyFunction(PyObject[] args, String[] kw);

The args array holds all argument values, whereas the kw array holds only the specified keys. The following is an example call of the previously mentioned MyFunction:

MyFunction(1, 2, D=4, C=9)

The args and kw arrays would then look like the following:

args = [1, 2, 4, 9]
kw = ["D", "C"]

Using these values requires some parsing, and the class org.python.core.ArgParser eases parsing these parameters into more useful forms. The ArgParser class has four constructors:

public ArgParser(String funcname, PyObject[] args,
         String[] kws, String p0)

public ArgParser(String funcname, PyObject[] args,
         String[] kws, String p0, String p1)

public ArgParser(String funcname, PyObject[] args,
         String[] kws, String p0, String p1, String p2)

public ArgParser(String funcname, PyObject[] args,
         String[] kws, String[] paramnames)

Each constructor requires the function's name as the first parameter. The function using ArgsParser should use the parameters PyObject[] args and String[] kw. These two objects become the second and third arguments to the ArgParser constructor. The remaining parameters are the list of arguments expected by a function. These can be separate args if there are three or fewer; otherwise, they are a String[]. Listing 1 shows a Java method that implements Jython's argument style with the help of ArgParser. Some example Jython methods and their Java method plus ArgParser counterparts are presented here to clarify:

Jython Method:
def test(A, B, C=2)

Java implementation:
public static PyObject test(PyObject[] args, String[] kws) {
  ArgParser ap = new ArgParser("test", args, kws, "A", "B", "C");
}

Jython Method:
def addContact(name, addr, ph=None)

Java implementation:
public static PyObject addContact(PyObject[] args, String[] kws) {
  ArgParser ap = new ArgParser("addContact", args, kws,
                 new String[] {"name", "addr", "ph"});
}

The parameter values are retrieved from the ArgParser instance with one of its get* methods. The get* methods listed here have either one or two parameters. Those with two parameters retrieve parameters with default values. Note that the position of the argument (pos) starts with position 0, not 1.

public String getString(int pos)

public String getString(int pos, String def)

public int getInt(int pos)

public int getInt(int pos, int def)

public PyObject getPyObject(int pos)

public PyObject getPyObject(int pos, PyObject def)

public PyObject getList(int pos)

For the Jython method:

def test(A, B, C=2)

The Java implementation, which allows for the default value, is this:

public static PyObject test(PyObject[] args, String[] kws) {
  ArgParser ap = new ArgParser("test", args, kws, "A", "B", "C");
  int A = ap.getInt(0);
  // or...
  // String A = ap.getString(0);
  // PyObject A = ap.getPyObject(0);
  // PyTuple A = (PyTuple)ap.getList(0);

  String B = ap.getString(1);
  // or...
  // int B = ap.getInt(1);
  // PyObject B = ap.getPyObject(1);
  // PyObject B = ap.getList(1);

  // here's the two argument version to allow for defaults
  int C = ap.getInt(2, 2);// first 2=position, second 2=default value
}

Importing Jython Modules in Java

Importing a Jython module in Java usually uses the __builtin__.__import__ function. The __builtin__ module is actually a class in the org.python.core package that you must first import with:

import org.python.core.__builtin__;
// or more frequently org.python.core.*

The __builtin__.__import__ function has four signatures:

public static PyObject __import__(String name)

public static PyObject __import__(String name, PyObject globals)

public static PyObject __import__(String name, PyObject globals,
                 PyObject locals)

public static PyObject __import__(String name, PyObject globals,
                 PyObject locals,PyObject fromlist)

Importing the random module from within Java would be as follows:

PyObject module = __builtin__.__import__(random);

Working with PyObjects

Calling Jython classes in Java, and writing Jython modules in Java requires extensive use of Jython's special methods. Special methods are those methods that begin and end with two underscores. Jython's flexible operation creates special needs for calling PyObjects from within Java.

Jython's dynamic operation means that the finding, getting, setting, and calling of attributes all happens through methods that provide opportunity to customize or extend each step. Two special methods often used from Java are __findattr__ and __finditem__. The __findattr__ method returns object attributes, and the __finditem__ method returns sequence or dictionary values designated by a key. Usages of these methods create the following signatures:

public PyObject __finditem__(PyObject key)
public PyObject __finditem__(int index)
public PyObject __finditem__(String key)

public PyObject __findattr__(String name)
public PyObject __findattr__(PyObject name)

The __finditem__ and __findattr__ methods that accept a String object as a parameter both require that the string object be interned. String literals are automatically interned, but otherwise, the string must be explicitly interned.

If you import the random module and intend to use the randint method, you cannot just call random.randint directly:

// This is not right
PyObject random = __builtin__.__import__("random");
random.randint(new PyInteger(10), new PyInteger(100));

Instead, you should use the __findattr__ method combined with the __call__ method:

PyObject random = __builtin__.__import__("random");
random.__findattr__("randint").__call__(new PyInteger(10), 
                    new PyInteger(20));

The shortcut method invoke exists to help in calling a method on a PyObject from Java. Specifically, the correct and generic way to call methods on all kinds of Jython mapping objects (such as PyDictionary and PyStringMap) is with the invoke method. Calling the invoke method on a PyObject's method is the equivalent of calling the following:

myPyObject.__getattr__(name).__call__(args, keywords)

Following are the different method signatures of the invoke method:

public PyObject invoke(String name)
public PyObject invoke(String name, PyObject arg1)
public PyObject invoke(String name, PyObject arg1, PyObject arg2)
public PyObject invoke(String name, PyObject[] args)
public PyObject invoke(String name, PyObject[] args, String[] keywords)

The different signatures respectively allow for differing sets of arguments passed to the invoked method, but what does not differ is that the first argument is a string representing the name of the PyObject's method to call. This name parameter must be an interned string. Remember that string literals are automatically interned.

The following example shows the creation of a PyDictionary and the retrieval of its keys from within Java using the invoke method. This example uses PyDictionary's empty constructor and subsequently adds values with the __setitem__ special method:

PyDictionary dct = new PyDictionary();
dct.__setitem__(new PyString("G"), new PyInteger(1));
dct.__setitem__(new PyString("D"), new PyInteger(2));
dct.__setitem__(new PyString("A"), new PyInteger(3));
dct.__setitem__(new PyString("E"), new PyInteger(4));
dct.__setitem__(new PyString("B"), new PyInteger(5));
PyObject keys = dct.invoke("keys");

The invoke method above uses the keys method of the PyDictionary object to retrieve its keys. Because the method name (keys) is a string literal in the previous example, it is automatically interned.

Looping through the keys returned in the previous example also requires special care. Because a PyObject's __len__ method may return wrong values, the only way to safely loop through Jython sequences in Java is to test for each index value until reaching a non-existing index. If we continue the dictionary keys example from above with a loop through those keys, the proper and generic way to loop through them is implemented as follows:

PyObject key;
for (int i = 0; (key = keys.__finditem__(i)) != null; i++) {
  System.out.println("K: " + key + ", V: " + dict.__getitem__(key));
}

Writing Jython Classes in Java

When a Java class emulates a Jython module, functions within that module are often implemented as static class members. Emulating a Jython class in Java can therefore be implemented as a static inner class; however, the ClassDictInit interface allows more flexibility in this regard. To emulate a Jython class with Java, it is best to subclass the most appropriate Py* class found in the org.python.core package and then implement all the special methods required for the desired type of class. All classes written in Java may subclass PyObject and implement the __findattr__, __setattr__, and __delattr__. A Mapping object would subclass PyDictionary or PyStringMap and implement __finditem__, __setitem__, __delitem__, and the associated mapping methods. The same is true for Jython's other data types.

Adding a Java Class as a Built-In Jython Module

When you write a Jython module in Java, you also have the option to designate this module as a built-in module. The registry key python.modules.builtin allows you to add modules to the list of built-in modules written in Java. The python.modules.builtin property is a comma-separated list of entries. The entries may appear in three forms. Table 1 shows each form of a module entry and an explanation of its use.

Table 1—Built-In Modules Entry Syntax and Description

Entry Syntax	Description
`name`	Just the name of the Java class. This assumes that the class is in the `org.python.modules` package. Adding this entry to the `python.modules.builtin` list allows you to use "import name" in Jython. If you wish to add the class `org.python.modules.jnios`, the would appear as follows: `python.modules.builtin = jnios`
`name:class`	This form requires a name and fully qualified class, separated by a colon. The name need not be the class name; it is simply the name Jython uses to refer to this class. If the name duplicates a pre-existing name, the pre-existing module is overridden. To add the class com.mycompany.python.agent as the name use the following: `python.modules.builtin = mycompanyagent:org.mycompany.python.agent`
`name:null`	This removes the module name from the list of built-in modules. To remove the `os` module, use the following: `python.modules.builtin = os:null`

The python.modules.builtin property is unique in how it is set. Currently setting this property with the –D command-line switch does not add a module to this built-in list. It must be set in the registry file, or in the post-properties (second arg) in the initialize method.

To add mymod as a module, compile it, and ensure that it is in the correct directory tree for its package and in the classpath. Then edit the registry file, adding the following:

python.modules.builtin = "mymod:org.python.demo.modules.mymod"

The name, or portion left of the colon, could be any legal identifier, but the class, or right side of the colon, must be a full package.class string uniquely identifying the appropriate class. Once a Java class is added as a built-in, it is available in Jython with just an import of its simple name. The mymod from the preceding example is available with the following:

>>> import mymod

This same mechanism for defining a built-in allows you to substitute a module for an existing built-in, or remove a built-in. All of this contributes to the ease of which Jython is extended and customized with Java.

About This Article

If you want more in-depth information on this topic, check out Chapter 9, "Embedding and Extending Jython in Java," of Jython for Java Programmers, by Robert Bill.