Processing Input and Output

This chapter is from the book

Core Java for the Impatient, 4th Edition

Learn More Buy

This chapter is from the book

This chapter is from the book 

Core Java for the Impatient, 4th Edition

Learn More Buy

9.3. HTTP Connections

You can read from a URL by using the input stream returned from URL.getInputStream method. However, if you want additional information about a web resource, or if you want to write data, you need more control over the process than the URL class provides. The URLConnection class was designed before HTTP was the universal protocol of the Web. It provides support for a number of protocols, but its HTTP support is somewhat cumbersome. When the decision was made to support HTTP/2, it became clear that it would be best to provide a modern client interface instead of reworking the existing API. The HttpClient provides a more convenient API and HTTP/2 support.

In the following sections, I provide a cookbook for using the HttpURLConnection class, and then give an overview of the API.

9.3.1. The URLConnection and HttpURLConnection Classes

To use the URLConnection class, follow these steps:

1. Get an URLConnection object:

   URLConnection connection = url.openConnection();

   For an HTTP URL, the returned object is actually an instance of HttpURLConnection.

2. If desired, set request properties:

   connection.setRequestProperty("Accept-Charset", "UTF-8, ISO-8859-1");

   If a key has multiple values, separate them by commas.

3. To send data to the server, call

   connection.setDoOutput(true);
   try (OutputStream out = connection.getOutputStream()) {
       // Write to out
   }

4. If you want to read the response headers and you haven’t called getOutputStream, call

   connection.connect();

   Then query the header information:

   Map<String, List<String>> headers = connection.getHeaderFields();

   For each key, you get a list of values since there may be multiple header fields with the same key.

5. Read the response:

   try (InputStream in = connection.getInputStream()) {
       // Read from in
   }

A common use case is to post form data. The URLConnection class automatically sets the content type to application/x-www-form-urlencoded when writing data to a HTTP URL, but you need to encode the name/value pairs:

URL url = ...;
URLConnection connection = url.openConnection();
connection.setDoOutput(true);
try (var out = new OutputStreamWriter(
        connection.getOutputStream())) {
    Map<String, String> postData = ...;
    boolean first = true;
    for (Map.Entry<String, String> entry : postData.entrySet()) {
        if (first) first = false;
        else out.write("&");
        out.write(URLEncoder.encode(entry.getKey(), "UTF-8"));
        out.write("=");
        out.write(URLEncoder.encode(entry.getValue(), "UTF-8"));
    }
}
try (InputStream in = connection.getInputStream()) {
    ...
}

9.3.2. The HTTP Client API

The HTTP client API provides another mechanism for connecting to a web server which is simpler than the URLConnection class with its rather fussy set of stages. More importantly, the implementation supports HTTP/2.

An HttpClient can issue requests and receive responses. You get a client by calling

HttpClient client = HttpClient.newHttpClient();

Alternatively, if you need to configure the client, use a builder API like this:

HttpClient client = HttpClient.newBuilder()
    .followRedirects(HttpClient.Redirect.ALWAYS)
    .build();

That is, you get a builder, call methods to customize the item that is going to be built, and then call the build method to finalize the building process. This is a common pattern for constructing immutable objects.

Follow the same pattern for formulating requests. Here is a GET request:

HttpRequest request = HttpRequest.newBuilder()
    .uri(URI.create("https://horstmann.com"))
    .GET()
    .build();

The URI is the “uniform resource identifier” which is, when using HTTP, the same as a URL. However, in Java, the URL class has methods for actually opening a connection to a URL, whereas the URI class is only concerned with the syntax (scheme, host, port, path, query, fragment, and so on).

When sending the request, you have to tell the client how to handle the response. If you just want the body as a string, send the request with a HttpResponse.BodyHandlers.ofString(), like this:

HttpResponse<String> response
    = client.send(request, HttpResponse.BodyHandlers.ofString());

The HttpResponse class is a template whose type denotes the type of the body. You get the response body string simply as

String bodyString = response.body();

There are other response body handlers that get the response as a byte array or a file. One can hope that eventually the JDK will support JSON and provide a JSON handler.

With a POST request, you similarly need a “body publisher” that turns the request data into the data that is being posted. There are body publishers for strings, byte arrays, and files. Again, one can hope that the library designers will wake up to the reality that most POST requests involve form data, file uploads, or JSON objects, and provide appropriate publishers.

Nowadays, the most common POST request body contains JSON, which you need to convert to a string. Then you can form the following request:

HttpRequest request = HttpRequest.newBuilder()
   .uri(URI.create(urlString))
   .header("Content-Type", "application/json")
   .POST(HttpRequest.BodyPublishers.ofString(jsonString))
   .build();

The book’s companion code has examples for posting form data and file uploads.

The HttpRequest.Builder class also has build methods for the less common PUT, DELETE, and HEAD requests.

Java 16 adds a builder for filtering the headers of an existing HttpRequest. You provide the request and a function that receives the header names and values, returning true for those that should be retained. For example, here we modify the content type:

HttpRequest request2 = HttpRequest.newBuilder(request,
      (name, value) -> !name.equalsIgnoreCase("Content-Type")) // Remove old content type
   .header("Content-Type", "application/xml") // Add new content type
   .build();

The HttpResponse object also yields the status code and the response headers.

int status = response.statusCode();
HttpHeaders responseHeaders = response.headers();

You can turn the HttpHeaders object into a map:

Map<String, List<String>> headerMap = responseHeaders.map();

The map values are lists since in HTTP, each key can have multiple values.

If you just want the value of a particular key, and you know that there won’t be multiple values, call the firstValue method:

Optional<String> lastModified = headerMap.firstValue("Last-Modified");

You get the response value or an empty optional if none was supplied.

The HttpClient is autocloseable, so you can declare it in a try-with-resources statement. Its close method waits for the completion of submitted requests and then closes its connection pool.

jdk.httpclient.HttpClient.log=all

jdk.httpclient.HttpClient.level=INFO