An in-depth piece exploring building a modular event-driven microservices architecture, using Spring and Orkes Conductor for orchestration:
Java is one of the pillars of the open-source world. Almost every Java project uses other open-source projects since no one wants to reinvent the wheel. However, many times it happens that we need a library for its functionality but we have no clue how to use it. We run into things like:
- What is it with all these “*Service” classes?
- How do I instantiate this, it takes too many dependencies. What is a “latch“?
- Oh, I put it together, but now it starts throwing IllegalStateException. What am I doing wrong?
The trouble is that not all library designers think about their users. Most think only about functionality, and features, but few consider how the API is going to be used in practice, and how the users’s code will look and be tested.
This article comes with a few pieces of advice on how to save our users some of these struggles – and no, it’s not through writing documentation. Of course, an entire book could be written on this subject (and a few have been); these are some of the key points I learned while working on several libraries myself.
This should be obvious but many times it isn’t. Before starting to write any line of code, we need to have a clear answer to some questions: what inputs are needed? what is the first class my user will see? do we need any implementations from the user? what is the output? Once these questions are clearly answered everything becomes easier since the library already has a lining, a shape.
This is maybe the most important topic. We have to make sure it’s clear what the user needs to provide to the library in order for it to do its work. In some cases this a very trivial matter: it could be just a String representing the auth token for an API, but it also might be an implementation of an interface, or an abstract class.
A very good practice is to take all the dependencies through constructors and to keep these short, with a few parameters. If we need to have a constructor with more than three or four parameters, then the code should clearly be refactored. And if methods are used to inject mandatory dependencies then the users will most likely end up with the third frustration described in the overview.
Also, we should always offer more than one constructor, give users alternatives. Let them work both with String and Integer or don’t restrict them to a FileInputStream, work with an InputStream, so they can submit maybe ByteArrayInputStream when unit testing etc.
For example, here are a few ways we can instantiate a Github API entry point using jcabi-github:
Github noauth = new RtGithub();
Github basicauth = new RtGithub("username", "password");
Github oauth = new RtGithub("token");
Simple, no hustle, no shady configuration objects to initialize. And it makes sense to have these three constructors, because you can use the Github website while logged out, logged in or an app can authenticate on your behalf. Naturally, some functionality won’t work if you are not authenticated, but you know this from the start.
As a second example, here is how we would work with charles, a web crawling library:
WebDriver driver = new FirefoxDriver();
Repository repo = new InMemoryRepository();
String indexPage = "http://www.amihaiemil.com/index.html";
WebCrawl graph = new GraphCrawl(
indexPage, driver, new IgnoredPatterns(), repo
It’s also quite self-explanatory, I believe. However, while writing this, I realize in the current version there is a mistake: all the constructors require the user to supply an instance of IgnoredPatterns. By default, no patterns should be ignored, but the user should not have to specify this. I decided to leave it like this here, so you see a counter example. I assume that you would try to instantiate a WebCrawl and wonder “What is it with that IgnoredPatterns?!”
Variable indexPage is the URL from where the crawl should start, driver is the browser to use (cannot default to anything since we do not know which browser is installed on the running machine). The repo variable will be explained below in the next section.
So, as you see in the examples, try to keep it simple, intuitive and self-explanatory. Encapsulate logic and dependencies in such a way that the user doesn’t scratch his head when looking at your constructors.
If you still have doubts, try to make HTTP requests to AWS using aws-sdk-java: you will have to deal with a so-called AmazonHttpClient, which uses a ClientConfiguration somewhere, then needs to take an ExecutionContext somewhere in between. Finally, you might get to execute your request and get a response but still have no clue what an ExecutionContext is, for instance.
This is mostly for libraries that communicate with the outer world. Here we should answer the question “how will the output be handled?”. Again, a rather funny question, but it’s easy to step wrong.
Look again at the code above. Why do we have to provide a Repository implementation? Why doesn’t the method WebCrawl.crawl() just return a list of WebPage elements? It’s clearly not the library’s job to handle the crawled pages. How should it even know what we would like to do with them? Something like this:
WebCrawl graph = new GraphCrawl(...);
List<WebPage> pages = graph.crawl();
Nothing could be worse. An OutOfMemory exception could happen out of nowhere if the crawled site happens to have, let’s say, 1000 pages – the library loads them all in memory. There are two solutions to this:
- Keep returning the pages, but implement some paging mechanism in which the user would have to supply the start and end numbers. Or
- Ask the user to implement an interface with a method called export(List<WebPage>), that the algorithm would call every time a max number of pages would be reached
The second option is by far the best; it keeps things simpler on both sides and is more testable. Think how much logic would have to be implemented on the user’s side if we went with the first. Like this, a Repository for pages is specified (to send them in a DB or write them on disk maybe) and nothing else has to be done after calling method crawl().
By the way, the code from the Input section above is everything that we have to write in order to get the contents of the website fetched (still in memory, as the repo implementation says, but it is our choice – we provided that implementation so we take the risk).
To summarize this section: we should never completely separate our job from the client’s job. We should always think what happens with the output we create. Much like a truck driver should help with unpacking the goods rather than simply throwing them out upon arrival at the destination.
Always use interfaces. The user should interact with our code only through strict contracts.
For example, in the jcabi-github library the class RtGithub si the only one the user actually sees:
Repo repo = new RtGithub("oauth_token").repos().get(
Issue issue = repo.issues()
.create("Example issue", "Created with jcabi-github");
The above snippet creates a ticket in the repository https://github.com/eugenp/tutorials. Instances of Repo and Issue are used, but the actual types are never revealed. We cannot do something like this:
Repo repo = new RtRepo(...)
The above is not possible for a logical reason: we cannot directly create an issue in a Github repo, can we? First, we have to login, then search the repo and only then we can create an issue. Of course, the scenario above could be allowed, but then the user’s code would become polluted with a lot of boilerplate code: that RtRepo would probably have to take some kind of authorization object through its constructor, authorize the client and get to the right repo etc.
Interfaces also provide ease of extensibility and backward-compatibility. On one hand, we as developers are bound to respect the already released contracts and on the other, the user can extend the interfaces we offer – he might decorate them or write alternative implementations.
In other words, abstract and encapsulate as much as possible. By using interfaces we can do this in an elegant and non-restrictive manner – we enforce architectural rules while giving the programmer freedom to enhance or change the behaviour we expose.
To end this section, just keep in mind: our library, our rules. We should know exactly how the client’s code is going to look like and how he’s going to unit test it. If we do not know that, no one will and our library will simply contribute in creating code that is hard to understand and maintain.
4. Third Parties
Keep in mind that a good library is a light-weight library. Your code might solve an issue and be functional, but if the jar adds 10 MB to my build, then it’s clear that you lost the blueprints of your project a long time ago. If you need a lot of dependencies you are probably trying to cover too much functionality and should break the project into multiple smaller projects.
Be as transparent as possible, whenever possible do not bind to actual implementations. The best example that comes to mind is: use SLF4J, which is only an API for logging – do not use log4j directly, maybe the user would like to use other loggers.
Document libraries that come through your project transitively and make sure you don’t include dangerous dependencies such as xalan or xml-apis (why they are dangerous is not for this article to elaborate).
Bottom line here is: keep your build light, transparent and always know what you are working with. It could save your users more hustle than you could imagine.
The article outlines a few simple ideas that can help a project stay on the line with regards to usability. A library, being a component that should find its place in a bigger context, should be powerful in functionality yet offer a smooth and well-crafted interface.
It is an easy step over the line and makes a mess out of the design. The contributors will always know how to use it, but someone new who first lays eyes on it might not. Productivity is the most important of all and following this principle, the users should be able to start using a library in a matter of minutes.