Introduction

The HttpBuilder-NG project is a modern Groovy DSL for making HTTP requests. It is usable with both Groovy and Java, though it requires Java 8 and a modern version of Groovy. It is built against Groovy 2.4.x, but it doesn’t make any assumptions about which version of Groovy you are using. The main goal of HttpBuilder-NG is to allow you to make HTTP requests in a natural and readable way.

History

HttpBuilder-NG was forked from the HTTPBuilder project originally developed by Thomas Nichols. It was later passed on to Jason Gritman who maintained it for several years, though it seems to have grown stagnant.

The original intent of HttpBuilder-NG was to fix a few bugs and add a slight enhancement to the original HTTPBuilder project. The slight enhancement was to make HTTPBuilder conform to more modern Groovy DSL designs; however, it was not possible to update the original code to have a more modern typesafe DSL while preserving backwards compatibility. I decided to just make a clean break and give the project a new name to make it clear that HttpBuilder-NG is basically a complete re-write and re-architecture.

Quick Start

If you are itching to get started, here’s what you need to do to get up and running.

Add Library to Project

First, you need the library in your project. If you are using Gradle, add the following dependency to your build.gradle file dependencies closure:

compile 'io.github.http-builder-ng:http-builder-ng-CLIENT:0.14.2'

or, for Maven add the following to your pom.xml file:

<dependency>
  <groupId>io.github.http-builder-ng</groupId>
  <artifactId>http-builder-ng-CLIENT</artifactId>
  <version>0.14.2</version>
</dependency>

where CLIENT is replaced with the client library name (core, apache, or okhttp).

Instantiate HttpBuilder

Next, you need to instantiate an instance of HttpBuilder to make requests on. If you are using Groovy:

import static groovyx.net.http.HttpBuilder.configure

def http = configure {
    request.uri = 'http://localhost:9876'
}

or, if you are using Java:

import static groovyx.net.http.HttpBuilder.configure;

def http = configure(config -> {
    config.getRequeset().setUri("http://localhost:9876");
});

A request.uri is the only required property, though other global and client configurations may be configured in the configure block.

The code shown will use the default client library based on the Java HttpUrlConnection; however, you can also use the Apache HttpComponents or the OkHttp client libraries. See the section on Client-Related Configuration for more details on switching client implementations.

Make Requests

Once you have the library and an instance of HttpBuilder you are ready to make requests. For Groovy:

String text = http.get(String){
    request.uri.path = '/name'
}

and for Java:

String text = http.get(String.class, cfg -> {
    cfg.getRequest().getUri().setPath("/name");
});

At the end of the User Guide there is an Examples section with some complete Groovy script examples.

Configuration

HttpBuilder configuration falls into two categories, the client configuration and the per-request configuration:

def request = HttpBuilder.configure {
    // Configure client or shared settings here (with HttpObjectConfig)
}.get {
    // Configure request here (with HttpConfig)
}

where in the example above, get could be replaced with any of the supported HTTP verbs (e.g. get, head, post, put, delete).

There are two primary configuration interfaces, HttpConfig and HttpObjectConfig. The former is used across all configuration methods, while the latter is used for the client-related configuration.

HttpConfig

The HttpConfig interface provides the public interface used for the HttpBuilder shared and per-verb configuration. It provides accessors for request and response configuration with the getRequest() and getResponse() methods.

Tip
Request and Response configuration configured in the configure method will be applied to all requests made on the HttpBuilder instance, unless the configuration is overridden in the request-specific configuration.

Request

The HttpConfig.getRequest() method returns an instance of HttpConfig.Request which may be used to configure various properties of a request with either the Groovy DSL or a Java Consumer.

URI

The setUri(String) method of HttpConfig.Request is the only required configuration property of an HttpBuilder instance since it defines the target URI for the request.

HttpBuilder.configure {
    request.uri = 'http://localhost:9191'
}

The URI may be extended in the request-specific configuration using the getUri() method, which returns an instance of groovyx.net.http.UriBuilder populated with the URI information from the client configuration. This is useful to change the path or query string information of the URI for a specific request, while leaving the base URI in the global configuration:

HttpBuilder.configure {
    request.uri = 'http://localhost:9191'
}.get {
    request.uri.path = '/person'
    request.uri.query = [name: 'Bob']
}
Cookies

HTTP Cookies may be configured on a request using one of the cookie configuration methods on the HttpConfig.Request interface:

  • cookie(String name, String value) - Adds a cookie with the specified name and value to the request.

  • cookie(String name, String value, Date expires) - Adds a cookie with the specified name, value and expiration date to the request (as Date).

  • cookie(String name, String value, LocalDateTime expires) - Adds a cookie with the specified name, value and expiration date to the request (as LocalDateTime).

Cookies may be configured globally or per-request:

HttpBuilder.configure {
    request.cookie('user-id', 'some-identifier')
}.head {
    request.cookie('user-session', 'SDF@$TWEFSDT', new Date()+30)
}
Encoders

Content encoders are used to convert request body content to a different format before handing it off to the underlying HTTP client. An encoder is implemented as a java.util.function.BiConsumer<ChainedHttpConfig,ToServer> function where the provided implementation of the ToServer provides the data. See the toServer(InputStream) method.

Encoders are configured in the request configuration using the encoder(String, BiConsumer<ChainedHttpConfig,ToServer>) or the encoder(Iterable<String>, BiConsumer<ChainedHttpConfig,ToServer>) method. The first parameter of each is the content type(s) that the encoder should be applied to, while the second parameter is the encoder function itself.

Say we wanted to be able to send Date objects to the server in a specific format as the request body:

HttpBuilder.configure {
    request.uri = 'http://locahost:1234/schedule'
    request.body = new Date()
    request.contentType = 'text/date-time'
    request.encoder('text/date-time'){ ChainedHttpConfig config, ToServer req->
        req.toServer(new ByteArrayInputStream("DATE-TIME: ${config.request.body.format('yyyyMMdd.HHmm')}".bytes))
    }
}.post()

Notice that a Groovy Closure is usable as a BiConsumer function. The Date object in the request is formatted as String, converted to bytes and pushed to the request InputStream. The same example could be written in Java as:

HttpBuilder.configure(config -> {
    config.getRequest().setUri("http://locahost:1234/schedule");
    config.setBody(new Date());
    config.setContentType("text/date-time")
    config.getRequest().encoder("text/date-time", new BiConsumer<ChainedHttpConfig,ToServer>(){
        public void accept(ChainedHttpConfig cfg, ToServer ts){
            String converted = new SimpleDateFormat("yyyyMMdd.HHmm").format(config.getRequest().getBody());
            ts.toServer(new ByteArrayInputStream(("DATE-TIME: " + converted).getBytes()));
        }
    });
}).post();

Some default encoders are provided:

  • CSV (when the com.opencsv:opencsv:3.8 library is on the classpath)

  • JSON (when either Groovy or the com.fasterxml.jackson.core:jackson-databind:2.8.1 library is on the classpath)

  • TEXT (with no additional libraries required)

  • XML (without any additional libraries)

  • Multipart (with client-specific implementations or when the JavaMail library is on the classpath)

Specific dependency versions are as of the writing of this document, see the project build.gradle dependencies block for specific optional dependency versions.

Authentication

The getAuth() method of HttpConfig.Request provides configuration access to the two methods of authentication supported: BASIC and DIGEST. Each is configured in a similar manner with the username and password to be sent with the request.

For BASIC:

HttpBuilder.configure {
    request.uri = 'http://localhost:10101'
    request.auth.basic 'admin', 'myp@$$w0rd'
}

For DIGEST:

HttpBuilder.configure {
    request.uri = 'http://localhost:10101'
    request.auth.digest 'admin', 'myp@$$w0rd'
}

There is nothing more to do on the client side.

Warning
Currently, the OkHttp client will only support DIGEST configuration in the configure method, not in the request-specific configuration methods - this is due to how the client configures DIGEST support internally.
Content

The HttpConfig.Request interface has a few methods related to the request content:

  • setAccept(Iterable<String>) and setAccept(String[]) - specifies the Accept header value.

  • setContentType(String) - specifies the Content-Type header value.

  • setCharset(Charset) - specifies the Accept-Charset header value.

  • setBody(Object) - specifies the body content for the request.

An example would be:

HttpBuilder.configure {
    request.uri = 'http://localhost:8675'
    request.contentType = 'text/plain'
    request.charset = Charsets.UTF_8
}.post {
    request.body = 'Let them eat content!'
}

Note that the body content and content-type come into play when configuring the request encoders; be sure that you have an encoder configured to handle the type of content data you are providing and that it renders the data properly to the request output.

Headers

Custom HTTP request headers may be configured directly using the getHeaders() method of the HttpConfig.Request instance. A Map<String,String> is returned which may be used to add new headers or modify headers already configured on the request:

HttpBuilder.configure {
    request.headers['Global-Header'] = 'header-for-all'
}.post {
    request.headers['Custom-Header'] = 'some-custom-value'
}

These configured headers will be appended to the default request headers sent by the request (somewhat client-specific).

Multipart

HttpBuilder-NG supports multipart request content such as file uploads, with either the generic MultipartEncoder or one of the client-specific encoders. For example, the OkHttpBuilder may use the OkHttpEncoders.&multipart encoder:

import groovyx.net.http.OkHttpBuilder
import groovyx.net.http.*

File someFile = // ...

OkHttpBuilder.configure {
    request.uri = 'http://example.com'
}.post {
    request.uri.path = '/upload'
    request.contentType = 'multipart/form-data'
    request.body = multipart {
        field 'name', 'This is my file'
        part 'file', 'myfile.txt', 'text/plain', someFile
    }
    request.encoder 'multipart/form-data', OkHttpEncoders.&multipart
}

which would POST the content of the file, someFile along with the specified name field to the server as a multipart/form-data request. The important parts of the example are the multipart DSL extension, which is provided by the MultipartContent class and aids in creating the upload content in the correct format. The multipart encoder is used to convert the request content into the multipart message format expected by a server. Notice that the encoder is specific to the OkHttpBuilder, which we are using in this case.

The available multipart encoders:

  • groovyx.net.http.CoreEncoders::multipart - a generic minimalistic multipart encoder for use with the core Java client or any of the others (requires the JavaMail API).

  • groovyx.net.http.OkHttpEncoders::multipart - the encoder using OkHttp-specific multipart encoding.

  • groovyx.net.http.ApacheEncoders::multipart - the encoder using Apache client specific multipart encoding.

The encoding of the parts is done using the encoders configured on the HttpBuilder executing the request. Any encoders required to encode the parts of a multipart content object must be specified beforehand in the request configuration.

Response

The HttpConfig.getResponse() method returns an instance of HttpConfig.Response which may be used to configure various properties of a request.

Status Handlers

The HttpConfig.Response interface provides five types of response status handlers.

The first three are the when methods. The when methods configure a handler (as a Groovy Closure or BiFunction<FromServer,Object,?>) to be called when the response status matches the specified status value (as a String, Integer, or HttpConfig.Status value). When the handler is called, it is executed and its return value is used as the return value for the request.

HttpBuilder.configure {
    request.uri = 'http://localhost:8181'
}.head {
    reponse.when(200){ FromServer fs ->
        'ok'
    }
}

In this example, if the request responds with a 200 status, the resulting value from the head call will be the String "ok".

The other two status handlers are the success and failure methods:

  • The success methods accept either a Groovy Closure or a BiFunction<FromServer,Object,?> as a handler. The handler is then configured as a when handler for response status code values less than 400.

  • The failure methods accept either a Groovy Closure or a BiFunction<FromServer,Object,?> as a handler, which is then configured as a when handler for response status code values greater than or equal to 400.

HttpBuilder.configure {
    request.uri = 'http://localhost:8181'
}.head {
    reponse.success { FromServer fs ->
        'ok'
    }
}

This example performs the same operation as the previous example, but uses the success method instead of the when method.

Exception Handlers

The main strategy for handling exceptions in the library, client implementations, or in the server response is:

  1. By default allow exceptions to propagate.

  2. If 1 is not feasible (because of interface restrictions, lambda restrictions, or too inconvenient), then exceptions should be wrapped in a TransportingException and re-thrown. The TransportingException is a signal to unwrap the exception before calling the exception handler.

  3. In the Builders wrap all request/response executions inside a try/catch block. In the catch block(s) call HttpBuilder.handleException() to handle the exception and use the value returned from that method as the return value for the request/response.

This should ensure that the original exception thrown makes it to the exception handler. Handlers may be chained in a manner similar to the success/failure handlers.

Exception handlers are configured on the HttpConfig configuration object using the exception(Closure) or exception(Function<Throwable,?>) method The value returned from the handler will be used as the result value of the request. Since there is no response body for the function to process, this usually means that the function should do one of three things: re-throw the exception or throw a wrapped version of the exception, return null, or return a predefined empty value.

HttpBuilder.configure {
    request.uri = 'http://localhost:10101'
}.get {
    request.uri.path = '/foo'
    response.exception { t->
         t.printStackTrace()
         throw new RuntimeException(t)
    }
}

The built-in exception method wraps the exception in a java.lang.RuntimeException (if it is not already of that type) and re-throws it.

Parsers

The response body content resulting form a request is parsed based on the response content-type. Content parsers may be configured using the HttpConfig.Response.parser(String, BiFunction<ChainedHttpConfig, FromServer, Object>) method, which takes a BiFunction and the response content type it is mapped to. The function (or Closure) accepts a ChainedHttpConfig object, and a FromServer instance and returns the parsed Object. If we had a server providing the current time as a response like DATE-TIME: MM/dd/yyyy HH:mm:ss we could request the time with the following code:

Date date = HttpBuilder.configure {
    request.uri = 'http://localhost:1234/currenttime'
}.get(Date){
    response.parser('text/date-time'){ ChainedHttpConfig cfg, FromServer fs ->
        Date.parse('MM/dd/yyyy HH:mm:ss', fs.inputStream.text)
    }
}

which would parse the incoming response and convert it to a Date object.

Some default parsers are provided:

  • HTML (when either the 'org.jsoup:jsoup:' or 'net.sourceforge.nekohtml:nekohtml:' library is on the classpath),

  • JSON (when either Groovy or the com.fasterxml.jackson.core:jackson-databind:2.8.1 library is on the classpath)

  • CSV (when the com.opencsv:opencsv:3.8 library is on the classpath)

  • XML (without any additional libraries)

  • TEXT (without any additional libraries)

Specific dependency versions are as of the writing of this document, see the project build.gradle dependencies block for specific optional dependency versions.

Headers

HTTP response headers are retrieved from the response using the FromServer.getHeaders() method. Some common headers are enriched with the ability to parse themselves into more useful types, for example:

headers.find { h-> h.key == 'Last-Modified' }.parse()   // ZonedDateTime
headers.find { h-> h.key == 'Allow' }.parse()           // List<String>
headers.find { h-> h.key == 'Refresh' }.parse()         // Map<String,String>

The parsing is provided using registered header implementations by header name. Currently, you cannot register your own and the supported header types are:

  • Access-Control-Allow-Origin→ ValueOnly

  • Accept-Patch→ CombinedMap

  • Accept-Ranges→ ValueOnly

  • Age→ SingleLong

  • Allow→ CsvList

  • Alt-Svc→ MapPairs

  • Cache-Control→ MapPairs

  • Connection→ ValueOnly

  • Content-Disposition→ CombinedMap

  • Content-Encoding→ ValueOnly

  • Content-Language→ ValueOnly

  • Content-Length→ SingleLong

  • Content-Location→ ValueOnly

  • Content-MD5→ ValueOnly

  • Content-Range→ ValueOnly

  • Content-Type→ CombinedMap

  • Date→ HttpDate

  • ETag→ ValueOnly

  • Expires→ HttpDate

  • Last-Modified→ HttpDate

  • Link→ CombinedMap

  • Location→ ValueOnly

  • P3P→ MapPairs

  • Pragma→ ValueOnly

  • Proxy-Authenticate→ ValueOnly

  • Public-Key-Pins→ MapPairs

  • Refresh→ CombinedMap

  • Retry-After→ HttpDate

  • Server→ ValueOnly

  • Set-Cookie→ MapPairs

  • Status→ ValueOnly

  • Strict-Transport-Security→ MapPairs

  • Trailer→ ValueOnly

  • Transfer-Encoding→ ValueOnly

  • TSV→ ValueOnly

  • Upgrade→ CsvList

  • Vary→ ValueOnly

  • Via→ CsvList

  • Warning→ ValueOnly

  • WWW-Authenticate→ ValueOnly

  • X-Frame-Options→ ValueOnly

All headers not explicitly typed are simply ValueOnly. The definitive list is in the source code of the groovyx.net.http.FromServer.Header class.

Multipart

While multipart responses are not supported by browsers, there is no restriction on sending them from a server and likewise the underlying HTTP clients have no problem handling them. HttpBuilder-NG does not provide a built-in multipart response decoder; however, using the JavaMail API, it is quite simple to implement one:

import javax.mail.BodyPart
import javax.mail.internet.MimeMultipart
import javax.mail.util.ByteArrayDataSource
import groovy.net.http.JavaHttpBuilder

MimeMultipart mimeMultipart = JavaHttpBuilder.configure {
    request.uri = // your server url
}.get(MimeMultipart){
    request.uri.path = '/download'
    response.parser(MULTIPART_MIXED[0]) { ChainedHttpConfig config, FromServer fs ->
        new MimeMultipart(new ByteArrayDataSource(fs.inputStream.bytes, fs.contentType))
    }
}

where the line:

new MimeMultipart(new ByteArrayDataSource(fs.inputStream.bytes, fs.contentType))

is where the JavaMail MimeMultipart class is used to parse the response content.

The JavaMail API support is optional, and requires that the JavaMail API library be on the classpath. Take a look at the ParsersSpec.groovy test case for the full implementation and an alternate implementation without using the JavaMail API.

HttpObjectConfig

The HttpObjectConfig interface is an extension of the HttpConfig interface, which provides additional client-level configuration options. These options should be configured in the HttpBuilder.configure methods, rather than in the per-verb configuration methods.

Client

The getClient() method of the HttpObjectConfig interface provides a means of applying client-specific configuration. Currently, there are only two supported configuration properties:

  • cookieVersion - the supported HTTP Cookie version used by the underlying clients. All three HttpBuilder implementations will support Cookies at version 0 by default, which is what the Java Servlet API accepts by default. This can be modified, but care must be taken to ensure that your server supports and accepts the configured version.

  • cookieFolder - the location for storing cookies that will persist after your application terminates. If no folder is specified an in memory cookie store and no cookies will be persisted after your application terminates. If cookies are found here then the cookies will be loaded prior to sending any requests to remote servers.

HttpBuilder.configure {
    client.cookieVersion = 0
    client.cookieFolder = new File('/tmp/cookies')
}

Execution

The getExecution() method of the HttpObjectConfig interface provides access to execution-specific configuration properties.

The first two properties are related to the concurrency of the HTTP clients, especially in the case where the asynchronous request methods are used:

  • executor - configures the java.util.concurrent.Executor to be used, by default a single-threaded Executor is used.

  • maxThreads - configures the maximum number of connection threads used by clients.

The second two properties are related to configuring SSL connections on the client:

  • sslContext - allows the specification of the javax.net.ssl.SSLContext that will be used.

  • hostnameVerifier - allows the specification of the javax.net.ssl.HostnameVerifier to be used.

The next section discusses the means of disabling SSL-related issues during connection.

Ignoring SSL Issues

During testing or debugging of HTTPS endpoints it is often useful to ignore SSL certificate errors. HttpBuilder-NG provides two methods of ignoring these issues. The first is via the configuration DSL using the groovyx.net.http.util.SslUtils::ignoreSslIssues(final HttpObjectConfig.Execution) method.

import groovyx.net.http.JavaHttpBuilder
import static groovyx.net.http.util.SslUtils.ignoreSslIssues

def http = JavaHttpBuilder.configure {
    ignoreSslIssues execution
    // other config...
}

Applying this configuration helper will set an SSLContext and HostnameVerifier which will allow/trust all HTTP connections and ignore issues. While this approach is useful, you may also need to toggle this feature at times when you do not, or cannot, change the DSL code itself; this is why the second approach exists.

If the groovyx.net.http.ignore-ssl-issues system property is specified in the system properties with a value of true, the ignoreSslIssues functionality will be applied by default.

Interceptors

The HttpObjectConfig interface allows the configuration of global request/response interceptors, which can perform operations before and after every request/response on the client. For example, if you wanted to make a POST request and return only the time elapsed during the request/response handling, you could do something like the following:

import static groovyx.net.http.HttpBuilder.configure
import static groovyx.net.http.HttpVerb.GET

long elapsed = configure {
    request.uri = 'https://mvnrepository.com/artifact/org.codehaus.groovy/groovy-all'
    execution.interceptor(GET) { cfg, fx ->
        long started = System.currentTimeMillis()
        fx.apply(cfg)
        System.currentTimeMillis() - started
    }
}.get(Long, NO_OP)

println "Elapsed time for request: $elapsed ms"

This interceptor on the GET requests will calculate the time spent in the actual request handling (the call to fx.apply(cfg) and return the elapsed time as the result of the request (ignoring the actual response content from the server). The displayed result will be something like:

Elapsed time for request: 865 ms

Using interceptors you can also modify the data before and after the apply() method is called.

The client-related configuration consists of five different methods. The first two methods configure(Closure) and configure(Consumer<HttpObjectConfig>) instantiate and configure an HttpBuilder object using the default client implementation, based on which HttpBuilder implementation is used. The HttpBuilder and JavaHttpBuilder will use the core Java client, while the ApacheHttpBuilder and OkHttpBuilder classes will use the Apache and OkHttp client implementations respectively.

The configuration Closure will delegate to an instance of HttpObjectConfig which provides the configuration DSL:

HttpBuilder.configure {
    request.uri = 'http://localhost:1234'
}

Likewise, the configuration Consumer<HttpObjectConfig> will have an instance of HttpObjectConfig passed into it for configuration:

HttpBuilder.configure(config -> {
    config.getRequest().setUri("http://localhost:1234");
});

The other three methods configure(Function<HttpObjectConfig,? extends HttpBuilder>), configure(Function<HttpObjectConfig,? extends HttpBuilder>, Closure), and configure(Function<HttpObjectConfig,? extends HttpBuilder>, Consumer<HttpObjectConfig>) have the same functionality as the other two methods mentioned above; however, they have an additional factory property which is used to provide the underlying HTTP client to be used. For the default HttpURLConnection-based implementation use the factory Function as:

HttpBuilder.configure({ c -> new JavaHttpBuilder(c) })

For the Apache-based builder, you would use the ApacheHttpBuilder in the factory, as:

HttpBuilder.configure({ c -> new ApacheHttpBuilder(c) })

Using the ApacheHttpBuilder requires the http-builder-ng-apache dependency to be added to your project. The third client implementation, OkHttpBuilder can be specified in the same manner (requiring the http-builder-ng-okhttp dependency).

The HttpBuilder class has request-related configuration methods for each of the supported request verbs, GET, HEAD, DELETE, POST, and PUT. Each request verb method has a synchronous and asynchronous version - the asynchronous versions are suffixed with Async, (e.g. headAsync) and they return a java.util.concurrent.CompletableFuture used to retrieve the eventual return value. Otherwise, the async methods are the same. Only the synchronous versions are discussed below.

An example of an async request would be:

CompletableFuture future = HttpBuilder.configure {
    request.uri = 'http://localhost:1234/somthing'
}.getAsync()

Object result = future.get()

Note the difference being the getAsync() call and the return type of CompletableFuture.

Each of the request verbs share the same configuration method forms. The examples in the following sections are for GET requests; however, they are representative of the available configurations. Also, at the end of the User Guide there is a collection of recipe-style example scripts for various common HTTP operations.

verb()

The no-argument method executes a request with the verb equivalent of the method name. The configuration for the request will come fully from the client-configuration. A simple get() example would be to determine who are the current astronauts in space:

@Grab('io.github.http-builder-ng:http-builder-ng-core:0.14.1')

import static groovyx.net.http.HttpBuilder.configure

def astros = configure {
    request.uri = 'http://api.open-notify.org/astros.json'
}.get()

println "There are ${astros.number} astronauts in space right now."

astros.people.each { p->
    println " - ${p.name} (${p.craft})"
}

which will print out something like:

There are null astronauts in space right now.
 - Sergey Rizhikov (ISS)
 - Andrey Borisenko (ISS)
 - Shane Kimbrough (ISS)
 - Oleg Novitskiy (ISS)
 - Thomas Pesquet (ISS)
 - Peggy Whitson (ISS)

verb(Closure)

The verb methods accepting a Closure or a Consumer<HttpConfig> executes a request with the verb equivalent of the method name. The configuration for the request will come from the merging of the client and request configurations. An example of a get(Closure) request would be to retrieve the current position if the ISS:

@Grab('io.github.http-builder-ng:http-builder-ng-core:0.14.1')

import static groovyx.net.http.HttpBuilder.configure

def iss = configure {
    request.uri = 'http://api.open-notify.org'
}.get {
    request.uri.path = '/iss-now.json'
}

println "The ISS is currently at lat:${iss.iss_position.latitude}, lon:${iss.iss_position.longitude}."

which will print out something like:

The ISS is currently at lat:32.4183, lon:7.8421.

The Java Consumer<HttpConfig>-based version of this would be similar to:

import static groovyx.net.http.HttpBuilder.configure;

Map<String,Object> iss = configure( cfg-> {
    cfg.getRequeste().setUri("http://api.open-notify.org");
}).get( cfg-> {
    cfg.getRequest().getUri().setPath('/iss-now.json');
});

String lat = iss.get("iss_position").get("latitude");
String lon = iss.get("iss_position").get("longitude");
System.out.println("The ISS is currently at lat:" + lat + ", lon:" + lon + ".");

verb(Class,Closure)

The verb methods accepting a Closure or Consumer<HttpConfig> along with a Class executes a request with the verb equivalent of the method name. The configuration for the request will come from the merging of the client and request configurations. The response content will be cast as the specified type if possible - a response parser may be required to convert the response to an appropriate type.

An example of this for a get(Integer,Closure) call would be to retrieve the number of people currently in space:

@Grab('io.github.http-builder-ng:http-builder-ng-core:0.14.1')

import static groovyx.net.http.HttpBuilder.configure
import groovy.json.JsonSlurper

int count = configure {
    request.uri = 'http://api.open-notify.org'
}.get(Integer){
    request.uri.path = '/astros.json'
    response.parser('application/json'){ cc, fs->
        new JsonSlurper().parse(fs.inputStream).number
    }
}

println "There are $count astronauts in space"

which will result in something like:

There are 6 astronauts in space

A similar configuration could be done with a Consumer<HttpConfig>, see the previous example for details.

Client Library Integration

Currently the HttpBuilder-NG library has three HTTP client implementations, one based on the HttpURLConnection class (called the "core" or "java" implementation), another based on the Apache Http Components (called the "apache" implementation) and the third based on OkHttp (the "okhttp" implementation); however, there is no reason other HTTP clients could not be used, perhaps the Google HTTP Java Client if needed.

A client implementation is an extension of the abstract HttpBuilder class, which must implement a handful of abstract methods for the handling the HTTP verbs:

protected abstract Object doGet(final ChainedHttpConfig config);
protected abstract Object doHead(final ChainedHttpConfig config);
protected abstract Object doPost(final ChainedHttpConfig config);
protected abstract Object doPut(final ChainedHttpConfig config);
protected abstract Object doDelete(final ChainedHttpConfig config);

There is also an abstract method for retrieving the client configuration, though generally this will be a simple getter:

protected abstract ChainedHttpConfig getObjectConfig();

And finally a method to retrieve the threading interface, again this is generally a getter for the configured thread executor.

public abstract Executor getExecutor();

Once the abstract contract is satisfied, you can use the new client just as the others, with your client in the factory function:

HttpBuilder.configure({ c -> new GoogleHttpBuilder(c); } as Function){
    request.uri = 'http://localhost:10101/foo'
}

The client extensions will reside in their own sub-projects that in turn depend on the core library. This allows the clients to have code and dependency isolation from other implementations and minimizes unused dependencies in projects using the library.

If you come up with something generally useful, feel free to create a pull request and we may be able to bring it into the project.

Examples

This section contains some stand-alone script examples of how you can use HttpBuilder. You can copy them and run them in the Groovy Shell or there are unit test versions for most of these examples in the ExamplesSpec.groovy file.

Resource Last Modified (HEAD)

Suppose you want to see when the last time a jar in the public Maven repository was updated. Assuming the server is exposing the correct date, you can use the Last-Modified header for the resource to figure out the date. A HEAD request works nicely for this, since you don’t care about the actual file content at this time, you just want the header information. HttpBuilder-NG makes this easy:

resource_last_modified.groovy
@Grab('io.github.http-builder-ng:http-builder-ng-core:0.15.0')

import static groovyx.net.http.HttpBuilder.configure
import groovyx.net.http.*

String uri = 'http://central.maven.org/maven2/org/codehaus/groovy/groovy-all/2.4.7/groovy-all-2.4.7.jar'
Date lastModified = configure {
    request.uri = uri
}.head(Date) {
    response.success { FromServer resp ->
        String value = FromServer.Header.find(
            resp.headers, 'Last-Modified'
        )?.value

        value ? Date.parse(
            'EEE, dd MMM yyyy  H:mm:ss zzz',
            value
        ) : null
    }
}

println "Groovy 2.4.7 last modified ${lastModified.format('MM/dd/yyyy HH:mm')}"

In the example we use the URL for the Groovy 2.4.7 jar file from the Maven Central Repository and execute a HEAD request on it and extract the Last-Modified header and convert it to a java.util.Date object and return it as the result. We end up with a resulting output line something like:

Groovy 2.4.7 last modified 06/07/2016 03:38

Alternately, using header parsing along with the java.time API, you can simplify the header conversion:

resource_last_modified_alt.groovy
@Grab('io.github.http-builder-ng:http-builder-ng-core:0.15.0')

import static groovyx.net.http.HttpBuilder.configure
import static java.time.format.DateTimeFormatter.ofPattern
import groovyx.net.http.*
import java.time.*

ZonedDateTime lastModified = configure {
    request.uri = 'http://central.maven.org/maven2/org/codehaus/groovy/groovy-all/2.4.7/groovy-all-2.4.7.jar'
}.head(ZonedDateTime) {
    response.success { FromServer resp ->
        resp.headers.find { h->
            h.key == 'Last-Modified'
        }?.parse(ofPattern('EEE, dd MMM yyyy  H:mm:ss zzz'))
    }
}

println "Groovy 2.4.7 (jar) was last modified on ${lastModified.format(ofPattern('MM/dd/yyyy HH:mm'))}"

which yields the same results, just with a cleaner conversion of the header data.

Scraping Web Content (GET)

Scraping content from web sites doesn’t seem to be a prevalent as it was years ago, but it’s a lot easier than it used to be. By default, text/html content is parsed with the JSoup HTML parser into a Document object:

web_scraping.groovy
@Grab('io.github.http-builder-ng:http-builder-ng-core:0.15.0')
@Grab('org.jsoup:jsoup:1.9.2')

import static groovyx.net.http.HttpBuilder.configure
import org.jsoup.nodes.Document

Document page = configure {
    request.uri = 'https://mvnrepository.com/artifact/org.codehaus.groovy/groovy-all'
}.get()

String license = page.select('span.b.lic').collect { it.text() }.join(', ')

println "Groovy is licensed under: ${license}"

In the example we make a GET request to the a secondary Maven repository to fetch the main entry page for the groovy-all artifact, which has the license information on it. The page is returned and parsed into a JSoup Document which we can then run a CSS selection query on to extract the license information and display it. You will end up with:

Groovy is licensed under: Apache 2.0

Sending/Receiving JSON Data (POST)

Posting JSON content to the server and parsing the response body to build an object from it is pretty common in RESTful interfaces. You can do this by creating a POST request with a "Content-Type" of application/json and a custom response parser:

sending_receiving_post.groovy
@Grab('io.github.http-builder-ng:http-builder-ng-core:0.15.0')

import static groovyx.net.http.HttpBuilder.configure
import static groovyx.net.http.ContentTypes.JSON
import groovyx.net.http.*

@groovy.transform.Canonical
class ItemScore {
    String item
    Long score
}

ItemScore itemScore = configure {
    request.uri = 'http://httpbin.org'
    request.contentType = JSON[0]
    response.parser(JSON[0]) { config, resp ->
        new ItemScore(NativeHandlers.Parsers.json(config, resp).json)
    }
}.post(ItemScore) {
    request.uri.path = '/post'
    request.body = new ItemScore('ASDFASEACV235', 90786)
}

println "Your score for item (${itemScore.item}) was (${itemScore.score})."

The custom response parser is needed to convert the parsed JSON data into your expected response data object. By default, the application/json response content type will be parsed to a JSON object (lazy map); however, in this case we want the response to be an instance of the ItemScore class. The example simply posts an ItemScore object (as a JSON string) to the server, which responds with the JSON string that it was provided.

The additional .json property call on the parsed data is to extract the JSON data from the response envelope - the site provides other useful information about the request. The end result is the following display:

Your score for item (ASDFASEACV235) was (90786).

Sending Form Data (POST)

Posting HTML form data is a common POST operation, and it is supported by HttpBuilder with a custom encoder, such as:

@Grab('io.github.http-builder-ng:http-builder-ng-core:0.15.0')

import static groovyx.net.http.HttpBuilder.configure
import static groovyx.net.http.ContentTypes.JSON
import groovyx.net.http.*
import static groovy.json.JsonOutput.prettyPrint

def result = configure {
    request.uri = 'http://httpbin.org'
    request.contentType = JSON[0]
}.post {
    request.uri.path = '/post'
    request.body = [id: '234545', label: 'something interesting']
    request.contentType = 'application/x-www-form-urlencoded'
    request.encoder 'application/x-www-form-urlencoded', NativeHandlers.Encoders.&form
}

println "You submitted: id=${result.form.id} and label=${result.form.id}"

which posts the specified form data to the http://httpbin.org/post which replies with JSON content containing the posted form data (along with some other information). The default JSON decoder is used to parse the JSON content into a multi-level Map, from which we can extract the data we sent.

Note that the NativeHandlers.Encoders.&form encoder is used to convert the provided map data into the encoded message before sending it to the server.

License

HttpBuilder-NG is licensed under the Apache 2 open source license.

Copyright 2017 David Clark

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.