Home > Articles > Programming > Java

The Basics of the MMAPI for Java Developers

In this chapter, you will learn the basics of the MMAPI and its implementation on Nokia devices. After studying the two example applications provided, you should be able to add multimedia playback and audio/video capturing features to your own application.
This chapter is from the book

Support multimedia playback and capturing in J2ME smart clients.

The successes of portable MP3 players and camera phones have proven the value of multimedia in mobile applications. Mobile users not only play back media contents on the go, but also share daily experiences via audio and video capturing. The J2ME Mobile Media API (MMAPI) enables multimedia applications on Java devices. It is supported on all Developer Platform 2.0 devices and many Developer Platform 1.0 devices. In this chapter, we cover the following topics:

  • Introduction to the MMAPI: covers the basics of the API and supported media formats.
  • Simple audio playback: uses a MIDI player example to show how to play back simple audio content from a local file or over the network. We discuss various player controls available on Series 40 devices.
  • Advanced media playback: goes beyond MIDI and tones to show how to play back media files with more complex audio and video formats.
  • Media capturing: uses a multimedia blog example to illustrate the use of the audio and video capturing API and how to share the captured data over the network.

The second half of this chapter ("Advanced Playback" and "Media Capturing") uses examples running on today's Series 60 devices rather than Series 40 devices. That helps the Series 40 developers to keep up with coming advances, since the MMAPI support is fast evolving with each new device release. It also helps developers to scale existing Series 40 applications up to higher end Series 60 devices. This chapter shows you the capabilities and programming techniques of the MMAPI on Nokia Series 40 and Series 60 Developer Platform devices.

Introduction to the MMAPI

The MMAPI is designed to support multiple media content types and data capturing mechanisms. It bears a lot of resemblance to the Generic Connection Framework (GCF) discussed in Chapter 7, "Data Connectivity." A subset of the MMAPI for simple audio playback is included in the MIDP 2.0 specification. All Nokia MIDP 2.0 devices, however, also implement the full MMAPI v1.1 (JSR 135) specification. The MMAPI features a generic factory class that instantiates media player objects from URI locator strings, InputStream objects, or DataSource objects. The device MMAPI implementation provides the concrete player classes for supported media formats. A player exposes some of its application-specific features via the Control interface. For example, a tone player has ToneControl, and a video capture player has VideoControl and RecordControl. We can interact with a player via its controls. Figure 9-1 shows the overall architecture of the MMAPI.

09fig01.gif

Figure 9-1 MMAPI architecture.

The Manager Class

The Manager class is the static factory class in the MMAPI. The createPlayer() method is the factory method used to create Player instances.

Create Player from URI Locators

The most versatile version of the createPlayer() method takes in a URI locator string to specify the network location of the media file, or the data capturing mode, or the in-memory empty device type.

static Player createPlayer (String locator)

In the MMAPI, three types of URI locator strings are supported.

  • For media playback, the URI could point to a media file available on a remote HTTP server. The server must return the correct MIME type in the HTTP header for the createPlayer() method to determine which player instance to instantiate. For example, the URI locator string http://host/sample.mid is typically associated with the audio/midi MIME type on HTTP servers, and it would result in an audio MIDI file player instance. The supported MIME types in Series 40 and Series 60 Developer Platform 2.0 devices are listed in Table 9-1.

    Table 9-1. MIME Types in the Nokia MMAPI Implementation (varies among devices)

    MIME Types

    Description

    audio/x-tone-seq

    tone sequence

    audio/wav

    wav audio format, but player cannot be created from InputStream using this MIME type

    audio/x-wav

    wav audio format

    audio/au

    au audio format

    audio/x-au

    au audio format, but player cannot be created from InputStream using this MIME type

    audio/basic

    raw audi format

    audio/amr

    amr audio format

    audio/amr-wb

    amr wb audio format

    audio/midi

    midi audio format

    audio/sp-midi

    extended midi format

    video/mp4

    Mpeg4 video format

    video/mpeg4

    mpeg4 video format, but player cannot be created from InputStream using this MIME type

    video/3gpp

    3gpp video format

    application/vnd.rn-realmedia

    real media video format

  • For media capture, the URL string takes the special format capture://audio for audio capture or capture://video for still-image capture on a camera phone. The video mode displays video from the camera's viewfinder until you instruct the program to take a snapshot. Media capture is not supported in current Series 40 devices but is available on Series 60 devices and will be available on future Series 40 devices.
  • For MIDI and tone sequence players, we can instantiate empty players in memory and then use MIDIControl and ToneControl objects to set content dynamically. The URI locator strings for such empty players are device://midi, which corresponds to the static value Manager.MIDI_DEVICE_LOCATOR, and device://tone, which corresponds to Manager.TONE_DEVICE_LOCATOR.

Create Player from Data Stream and MIME Type

The URI locator–based approach is simple and powerful. However, this approach relies on the server to provide the correct MIME types. If the server is configured incorrectly, or if the network protocol does not support MIME metadata (e.g., Unix socket), we cannor create the correct player. More importantly, not all media data is available over the network. We might need to play back locally stored media data, such as files bundled in the JAR file or data arrays stored in the RMS store. To address the above issues, we need a way to assign any MIME type programmatically to any data input stream available on the device. The second version of the createPlayer() method allows us to do just that.

static Player createPlayer (InputStream is, String type)

Refer to Table 9-1 for the list of supported MIME types. Please note that each individual device supports only a subset of those types. For example, most current Series 40 devices support only the audio/x-tone-seq, audio/midi, and audio/sp-midi MIME types. More detailed information is available in the Mobile Media API Technical Note published on the Forum Nokia Web site. If a null value is passed as the MIME type parameter, the implementation should try to figure out the media type based on its content. If the actual data in the InputStream is not encoded in the specified MIME format, or if the implementation cannot determine the media format when a null type parameter is passed, a MediaException may be thrown at runtime. We cannot instantiate data capture players with this method.

Create Player from DataSource

The third version of the createPlayer() method takes in a DataSource object to create a player. The DataSource class defines several abstract life cycle methods, which allow the users to specify how to connect to a custom data source and start or stop data transfer. A DataSource instance contains one or multiple SourceStream objects, which manage the actual media data. The SourceStream is different from the InputStream in the following aspects.

  • SourceStream supports an API for random seeking that is required by some custom media data protocols.
  • SourceStream defines abstract methods to support the concept of transfer size that is more suited for frame-delimited data (e.g., video).

Both the DataSource and SourceStream classes are abstract. They provide a framework for users to extend the MMAPI to support custom media data protocols. They are rarely used. We do not cover them in detail in this chapter.

Other Manager Methods

In addition to creating new player instances, we can use the Manager class to query the supported media types and protocols in this MMAPI implementation. Check out the MediaPlayer example later in this chapter to see those methods in action.

// Returns the supported media types for a given protocol
static String [] getSupportedContentTypes (String protocol)
// Returns the supported protocols for a given media type
static String [] getSupportedProtocols (String type)

The Manager class can also play tones directly to the device's speaker. This call does not block.

static void playTone (int tone, int duration, int vol)

The duration argument is the duration of the sound in milliseconds; the vol argument is the playback volume from 0 to 100; the tone argument takes a value from 0 to 127. The relationship between the tone value and the resultant frequency is as follows:

tone = 12 * log2 (freq/220) + 57

For example, the MIDI tone value 69 corresponds to the frequency of 440 Hz, which is the musical note A4. Table 9-2 shows musical notes and their corresponding frequencies and MIDI tone values. To get notes that are one octave higher or lower, we can add or subtract 12 from the MIDI tone values. Or, we can double or half the frequencies.

Table 9-2. Musical Notes and Their Corresponding Frequencies and MIDI Tone Values

Note

Frequency (Hz)

MIDI Tone

A4

440.00

69

A#

466.16

70

B

493.88

71

C

523.25

72

C#

554.36

73

D

587.33

74

D#

622.25

75

E

659.25

76

F

698.45

77

F#

739.98

78

G

783.99

79

G#

830.60

80

System Properties

The MMAPI specification leaves a lot of flexibility to implementers. For example, Nokia can decide what features and encoding types to support on each MMAPI-compatible device. As we have discussed, the Manager.getSupportedContentTypes() and Manager.getSupportedProtocols() static methods help us to determine the capabilities of the MMAPI implementation. In addition, the MMAPI implementation provides a number of system properties that can be retrieved via the System.getProperty() static method. Those properties give us information about the MMAPI implementation. Table 9-3 describes those properties and their values on Nokia 6230 (Series 40) and Nokia 6600 (Series 60) devices. The MediaPlayer example later in this chapter provides a utility to query MMAPI implementation capabilities based on the above methods and system properties.

Table 9-3. System Properties in the Nokia MMAPI Implementation

System Properties

Description

Nokia 6230

Nokia 6600

supports.mixing

Query for whether audio mixing is supported.

false

false

supports.audio.capture

Query for whether audio capture is supported.

false

true

supports.video.capture

Query for whether video capture is supported.

false

true

supports.recording

Query for whether recording is supported.

false

true

audio.encodngs

The string returned specifies the supported capture audio formats.

 

encoding = pcm
encoding = ulaw
encoding = alaw
encoding = wav

video.encodings

The string returned specifies the supported capture video formats (video recording).

 

 

video.snapshot.encodings

Supported video snapshot formats for the VideoControl.getSnapshot() method.

 

encoding = jpeg
encoding = bmp
encoding = png

microedition.media.version

Returns 1.1 for an implementation that is compliant with MMAPI v1.1.

1.1

1.0

streamable.contents

Returns formats that can be streamed. No streaming format is supported at this time.

 

 

Player

The MMAPI specification declares the Player interface, which specifies the common behavior of all Player implementation classes provided by MMAPI implementers (e.g., Nokia) to handle different media types. The most important attribute of a Player is its life cycle states.

Player Life Cycle

A Player object can have the following states. Figure 9-2 illustrates the state transitions.

  • CLOSED: The player has released most of its resources, and it can never be used again. We can change the player from any other state to the closed state by calling the Player.close() method.
  • UNREALIZED: The player object has just been instantiated in the heap memory. It has not allocated any resources.
  • REALIZED: If the Player.realize() method is called in a unrealized state, the player acquires required media resources and moves itself to the realized state. For example, if the player plays a remote media file over the HTTP network, the entire file is downloaded during the realizing process.
  • PREFETCHED: If the Player.prefetch() method is called, the player performs a number of potentially time-consuming startup tasks and moves itself to the prefetched state. For example, during the prefetching process, the player acquires controls over the camera, the speaker, or other exclusive resources. The prefetching could fail if another program is already using some of those recourses. If the failure happens, we can call prefetch() later again on the same player. Once the player is prefetched, it can be started without further delay. Theoretically, we should move a prefetched player back to realized state by calling Player.deallocate(), but this method has not been implemented in Nokia devices.
  • STARTED: By calling the Player.start() method, we can start the player, which starts the media playback or starts the capture player. Once the player is started, we can also call the Player.stop() method to stop it and return it to the prefetched state. A stopped player can be started again, and it will resume playing from the point at which it was stopped.
09fig02.gif

Figure 9-2 Player states.

PlayerListener

We can listen for the player's events by registering PlayerListener objects to a player instance. The PlayerListener interface declares only one method, playerUpdate(), which is invoked every time the registered player receives an event. The caller Player object passes the event and any application-specific data. Developers decide how to respond to the event by implementing this method.

void playerUpdate (Player player,
                     String event, Object data)

The event strings are defined as static variables in the PlayerListener interface. Most of them are self-explanatory: BUFFERING_STARTED, BUFFERING_STOPPED, CLOSED, DEVICE_AVAILABLE, DEVICE_UNAVAILABLE, DURATION_UPDATED, END_OF_MEDIA, ERROR, RECORD_ERROR, RECORD_STARTED, RECORD_STOPPED, SIZE_CHANGED, STARTED, STOPPED, STOPPED_AT_TIME, and VOLUME_CHANGED. Following are a couple of points to notice:

  • Player state changes have their corresponding events, such as CLOSED, STARTED, and STOPPED. The player life cycle method always returns immediately, and we can process state changes asynchronously.
  • A player could be stopped under several conditions. The END_OF_MEDIA event occurs when the entire media content is played back. The STOPPED_AT_TIME event occurs when the player is stopped at a preset time in a StopTimeControl (discussed later). The STOPPED event occurs only when the player's stop() method is invoked.
  • The DEVICE_UNAVAILABLE event occurs when there is an incoming call. The DEVICE_AVAILABLE event occurs when the call is ended.

The Player class provides methods to register and remove the PlayerListener objects.

void addPlayerListener (PlayerListener listener)
void removePlayerListener (PlayerListener listener)

Other Methods in the Player Interface

The Player class supports methods to query the status of the current media file.

String getContentType ()
long getDuration ()
long getMediaTime ()
int getState ()
TimeBase getTimeBase ()

The following methods set how many times the player will loop and play the content, the media time of the current play position, and a new TimeBase to synchronize this player with another one. Please note that the current Series 40 devices only support the system time base.

void setLoopCount (int count)
long setMediaTime (long now)
void setTimeBase (TimeBase master)

Control

The Control interfaces in MMAPI allow developers to control aspects of media-specific players programmatically. The same class can implement multiple Control interfaces for maximum API flexibility. Since the Player interface inherits from the Controllable interface, every Player class implements the following methods, which return the Control objects supported by this particular Player class.

Control getControl (String type)
Control [] getControls ()

Control objects are identified by the type strings. For example, the following code retrieves a VolumeControl object from an audio player and then adjusts the volume level.

VolumeControl vc = player.getControl ("VolumeControl");
vc.setLevel (50);
player.start ();

The MMAPI defines many player controls. However, only a subset of them is supported on the current Nokia devices. Table 9-4 lists the controls supported by different types of players on Nokia devices. Now we have covered the basics of the MMAPI; in the next two sections, we give concrete examples to show how to use the API.

Table 9-4. Players and Controls on Nokia MMAPI Implementation

MIME Types

Series 40 Controls

Series 60 Controls

audio/x-tone-seq

ToneControl, TempoControl, RateControl, PitchControl, VolumeControl, StopTimeControl

VolumeControl, StopTimeControl, ToneControl

audio/wav, audio/au, audio/amr

n/a

VolumeControl, StopTimeControl, RecordControl

audio/x-wav, audio/basic, audio/x-au, audio/amr-wb

n/a

VolumeControl, StopTimeControl

audio/midi, audio/sp-midi

MIDIControl, TempoControl, RateControl, PitchControl, VolumeControl, StopTimeControl

 

video/mp4, video/mpeg4, video/3gpp, application/vnd_rn-realmedia

n/a

VolumeControl, StopTimeControl, VideoControl

Video Capture

n/a

VideoControl, StopTimeControl

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020