Jackson Mix-in Annotations
Jackson has a feature called Mix-in annotations. With this feature, we can write cleaner code for the domain classes. Imagine we have domain classes like below.
// package com.example
public interface Item {
ItemType getType();
}
@Value
public class FooItem implements Item{
@NonNull String fooId;
@Override
public ItemType getType() {
return ItemType.FOO;
}
}
When implement these domain classes, no Jackson annotation is put on them.
To support serialization and deserialization with Jackson for these classes, add “Mix-in” classes, for example in a
separate package called com.example.jackson
.
Add ItemMixin
below to let Jackson be able to serialize and deserialize Item
and its subclasses.
@JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.PROPERTY,
property = "type")
@JsonSubTypes({
@JsonSubTypes.Type(value = FooItem.class, name = "FOO")
// ...
})
public abstract class ItemMixin {
}
// merge the Jackson annotations in the Mix-in class into the Item class,
// as if these annotations are in the Item class
objectMapper.addMixIn(Item.class, ItemMixin.class);
String json = "[{\"fooId\": \"1\", \"type\": \"FOO\"}]";
List<Item> items = mapper.readValue(json, new TypeReference<List<Item>>(){});
Note that FooItem
is implemented as an immutable class using @Value
from Lombok.
With @Value
annotated, FooItem
has no default constructor, which makes Jackson unable to serialize it by default.
Add FooItemMixin
below to fix it.
@JsonIgnoreProperties(value={"type"}, allowGetters=true)
abstract class FooItemMixin {
@JsonCreator
public FooItemMixin(@JsonProperty("fooId") String fooId) {
}
}
With help of Mix-in annotations, the domain classes don’t have to be compromised for Jackson support. All Jackson relevant annotations are in separate Mix-in classes in a separate package. Further, we could provide a simple Jackson module like below.
@Slf4j
public class MixinModule extends SimpleModule {
@Override
public void setupModule(SetupContext context) {
context.setMixInAnnotations(Item.class, ItemMixin.class);
context.setMixInAnnotations(FooItem.class, FooItemMixin.class);
log.info("module set up");
}
}
The consumers of the domain classes can simple register this module to take in all the Mix-in annotations.
objectMapper.registerModule(new MixinModule());
Jackson Mix-in helps especially if the domain classes are from a third party library. In this case, the source of the domain classes cannot be modified, using Mix-in is more elegant than writing custom serializers and deserializers.
Which Accessor Style? "Fluent" vs. "Java Bean"
Basically, there are two styles of accessor naming convention in Java.
One is the traditional “Java Bean” style.
public class Item {
ItemType getType();
}
Another is called the “fluent” style.
public class Item {
ItemType type();
}
The fluent style saves some typing when writing code, also makes code a bit less verbose, item.type()
.
For example, the Lombok library supports this fluent style.
lombok.accessors.fluent = [true | false]
(default: false)
If set to true, generated getters and setters will not be prefixed with the bean-standard ‘get, is or set; instead, the methods will use the same name as the field (minus prefixes).
Which style is better?
Actually, the verbose Java Bean style, is the better one.
It’s because a lot of third party libraries are acknowledging the Java Bean style.
For example, if Item
is going to be serialized by the Jackson library as a JSON string, the fluent style wouldn’t
work out of the box.
Also, most of DTO mapping libraries are using the Java Bean style too.
Therefore, using the standard Java Bean accessor style save effort when integrate our classes with other libraries and frameworks.
With the Lombok library and the auto-completion in modern IDEs, the Java Bean style doesn’t necessarily mean more typing.
The contains() Method in Java Collection Is Not "Type Safe"
Currency currency;
//...
if(currency.getSupportedCountries().contains(country)) {
//...
}
The Currency.getSupportedCountries()
returns a Collection
. Originally, the returned Collection
was Collection<Country>
.
The country
object in the above if-condition was of type Country
. The program has been well tested and worked as expected.
However, due to whatever reason, the getSupportedCountries()
is refactored to return a Collection<String>
.
The Java compiler complains nothing about the refactor. But the if-condition now is never true
in any cases, since
the equals()
method of String
has no idea about the equality with Country
and vice versa.
A bug!
It’s hard to detect this kind of bug, if the code is not well covered by unit tests or end-to-end tests.
In this sense, the contains()
method in Java Collection is not type safe.
How to Avoid
First, never change the behavior of an API when refactor.
In the above case, the signature of the getSupportedCountries()
API has changed.
This is a breaking change, which usually causes the client code fails to compile.
Unfortunately, in above case the client code doesn’t fail fast in the compile phase.
It’s better to add new API like getSupportedCountryCodes()
which returns a Collection<String>
, and @Deprecated
the old API, which can be further deleted some time later.
Second, make code fully covered by test cases as much as possible. Test cases can detect the bug earlier in the test phase.
Why contains() Is Not Generic
Why contains()
is not designed as contains(E o)
, but as contains(Object o)
?
There are already some discussion on this design in StackOverflow, like this one
and this one.
It’s said it’s no harm to let methods like contains()
in Collection
be no generic.
Being no generic, the contains()
can accept a parameter of another type, which is seen as a “flexible” design.
However, the above case shows that this design does have harm and cannot give developers enough confidence.
A method accepting a parameter of Object
means it accepting any type, which is too “dynamic” for a “static” language.
Another question is why a static language needs a “root” Object
?
Run Multiple Gradle Sub Project Tasks
Use the brace expansion mechanism of Bash, multiple sub project tasks can be run in the command line like below.
$ ./gradlew sub-prj-1:{tasks,help}
After brace expansion, it is same as below, but saves some typing.
$ ./gradlew sub-prj-1:tasks sub-prj-1:help
More examples.
$ ./gradlew sub-prj-{1,2}:test
# same as
# ./gradlew sub-prj-1:test sub-prj-2:test
$ ./gradlew sub-prj-{1,2}:{clean,build}
# same as
# ./gradlew sub-prj-1:clean sub-prj-1:build sub-prj-2:clean sub-prj-2:build
The if-else Control Flow Using Optional
Sometimes you may want to write the if-else control flow based on an Optional
object.
For example, an API from a third party library declares Optional
as its return type.
You need to compose an if-else like control flow using that Optional
object.
Of course, it can be done by testing isPresent()
in a traditional if-else statement.
var itemOpt = service.getItem(itemId);
if (itemOpt.isPresent()) {
addToOrder(itemOpt.get());
} else {
log.info("missing item {}", itemId);
sendOutItemMissedEvent();
}
The above code doesn’t take any advantage of Optional
.
Actually, since Java 9 the ifPresentOrElse(Consumer<? super T>, Runnable)
method can be used to implement such control flow, which is a bit more elegant.
service.getItem(itemId).ifPresentOrElse(
item -> {
addToOrder(item);
},
() -> {
log.info("missing item {}", itemId);
sendOutItemMissedEvent();
});
Print Exception Together With Parameterized Log Messages
Often when use SLF4J, you may wonder if an exception object can be printed in the same parameterized log message together with other objects. Actually SLF4J supports it since 1.6.0.
log.error("failed to add item {}", "item-id-1", new RuntimeException("failed to add item"));
The SLF4J call above prints log message like below.
13:47:16.119 [main] ERROR example.TmpTest - failed to add item item-id-1
java.lang.RuntimeException: failed to add item
at example.TmpTest.logErrorTest(TmpTest.java:10)
...
...
So it actually doesn’t need to log as below.
log.error("failed to add item {}", "item-id-1");
log.error("exception details:", new RuntimeException("failed to add item"));
Here is what the official FAQ says.
The SLF4J API supports parametrization in the presence of an exception, assuming the exception is the last parameter.
Use the first style to save one line.
SLF4J doesn’t clearly state this behavior in the Javadoc of Logger.error(String format, Object... arguments)
(at least not in the 1.7.26 version). Maybe if a new method like below is added to Logger
, programmer would
find out this feature more easily.
public void error(String format, Throwable t, Object... arguments);
Process Substitution
From man bash
, for “process substitution” it says
It takes the form of <(list) or >(list)
Note: there is no space between <
or >
and (
.
Also, it says,
The process list is run with its input or output connected to a FIFO or some file in /dev/fd. The name of this file is passed as an argument to the current command as the result of the expansion.
The <(list)
Form
$ diff <(echo a) <(echo b)
1c1
< a
---
> b
Usually the diff
takes two files and compares them.
The process substitution here, <(echo a)
, creates a file in /dev/fd
, for example /dev/fd/63
.
The stdout of echo a
command is connected to /dev/fd/63
.
Meanwhile, /dev/fd/63
is used as an input file/parameter of diff
command.
Similar for <(echo b)
.
After Bash does the substitution, the command is like diff /dev/fd/63 /dev/fd/64
.
In diff
’s point of view, it just compares two normal files.
In this example, one advantage of process substitution is eliminating the need of temporary files, like
$ echo a > tmp.a && echo b > tmp.b \
&& diff tmp.a tmp.b \
&& rm tmp.{a,b}
The >(list)
Form
$ echo . | tee >(ls)
Similar, Bash creates a file in /dev/fd
when it sees >(ls)
.
Again, let’s say the file is /dev/fd/63
.
Bash connects /dev/fd/63
with stdin of ls
command, also the file /dev/fd/63
is used as a parameter of
tee
command.
The tee
views /dev/fd/63
as a normal file.
tee
writes content, here is .
, into the file, and the content will “pipe” into the stdin of ls
.
Compare with Pipe
Pipe, cmd-a | cmd-b
, basically just passes stdout of the command on the left to the stdin of the command on the right.
Its data flow is restricted, which is from left to right.
Process substitution has more freedom.
# use process substitution
$ grep -f <(echo hello) file-a
hello
# use pipe
$ echo hello | xargs -I{} grep {} file-a
hello
And for commands like diff <(echo a) <(echo b)
, it’s not easy to be done by pipe.
More Examples
$ paste <(echo hello) <(echo world)
hello world
How it works
From man bash
,
Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files.
Read more about named pipe and /dev/fd.
For /dev/fd
,
The main use of the /dev/fd files is from the shell. It allows programs that use pathname arguments to handle standard input and standard output in the same manner as other pathnames.
In my OS (Ubuntu), the Bash uses /dev/fd
for process substitution.
$ ls -l <(echo .)
lr-x------ 1 user user 64 12月 19 11:19 /dev/fd/63 -> pipe:[4926830]
Bash replaces <(echo .)
with /dev/fd/63
.
The above command is like ls -l /dev/fd/63
.
Or find the backing file via,
$ echo <(true)
/dev/fd/63
(After my Bash does the substitution, the command becomes echo /dev/fd/63
,
which outputs /dev/fd/63
.)
Deploy a Rails App with Docker
This site is deployed using Docker containers on a small DigitalOcean Droplet.
Since I never have a private Docker registry, the original deploy flow was,
- push the source code to the remote Git repository
- on the Droplet, fetch the source code from the Git remote
- build an Docker image from the source code
- start the Docker container
Since some time ago, the docker build
has kept failing on the bundle install
instruction on the Droplet.
During bundle install
, the Bundler fails to build the sassc
gem.
The Bundler complains that,
virtual memory exhausted: Cannot allocate memory
, while building the sassc
gem.
A workaround I tried was,
- build the Docker image on a more powerful machine, like the local dev machine
- save the image into a tar archive (
docker save img-foo | bzip2 > foo.img.bz
) - copy the tar to the Droplet
- load the image from the tar (
bunzip2 foo.img.bz && docker load -i foo.img
) - start the Docker container
One problem of the workaround was the volume to transfer is a bit large (the image is around 1G before compression, and 370M after compression). It was time consuming over a slow network.
Current Deploy Workflow
Actually, Bundler supports fetching gems from local cache, not only from rubygems.org
.
Using bundle package --all
can,
Copy all of the .gem files needed to run the application into the vendor/cache directory.
In the future, when running bundle install, use the gems in the cache in preference to the ones on rubygems.org.
The sassc
gem is built on the local machine while bundle package
, and put into vendor/cache
directory.
Now, the deploy flow is like,
- run
bundle package --all
if the application’s dependencies change - check in
vendor/cache
folder and push it to the Git remote - on the Droplet, fetch the source code from the Git remote
- build an Docker image from the source code
- start the Docker container
The Dockerfile
is like,
USER app
RUN mkdir -p /home/app/source-code
WORKDIR /home/app/source-code
COPY vendor/cache ./vendor/cache/
COPY Gemfile* ./
# use RVM, so "bash -lc ..."
RUN bash -lc 'bundle install --deployment --without development test'
# copy the app source code
COPY . ./
Other Notes
If a gem is located at a particular git repository using the :git
parameter, like
gem "rails", "2.3.8", :git => "https://github.com/rails/rails.git"
Then the --all
option should be used in bundle package
.
Some gems are built using CMake.
When run bundle package --all
, CMakeCache.txt
files are created and stored in vendor/cache
.
The problem with CMakeCache.txt
files is that they use hardcoded paths of the machine runs bundle package --all
.
When build the Docker image in the remote server, the hardcoded paths fail the bundle install
.
Therefore, in the .dockerignore
file, add the following line.
vendor/cache/*/CMakeCache.txt
Java Generics, Bounded Wildcards and PECS
With a class hierarchy below,
class Fruit {
public void hello() {}
}
class Apple extends Fruit {}
class Banana extends Fruit {}
Lower Bounded Wildcards and Upper Bounded Wildcards
? super Fruit
is called lower bounded wildcard.
? extends Fruit
is called upper bounded wildcard.
Usually a class hierarchy is illustrated like,
Object
|
Fruit
/ \
Apple Banana
The base class is placed upper, subclasses placed lower.
For ? super Fruit
, the lower bound of the type argument is determined, which is Fruit
, so it’s called lower bounded wildcard.
Similar, for type argument ? extends Fruit
, its upper bound is fixed, so it’s called upper bounded wildcard.
”? super Fruit”
void func(List<? super Fruit> fruits) {
fruits.add(new Fruit()); // ok
fruits.add(new Apple()); // ok
fruits.add(new Banana()); // ok
// Error: Cannot resolve method 'hello'
fruits.get(0).hello();
}
The ?
in the declaration List<? super Fruit>
means fruits
is a List
of an unknown type.
The ? super Fruit
means the unknown type is limited to be Fruit
or its super class (Object
in this case).
When invoke func(List<? super Fruit> )
, List<Fruit>
or List<Object>
can be passed as the argument.
All the fruits.add(...)
in the above code is ok, because no matter what type actually the type argument is
Fruit
, Apple
, and Banana
are the subtypes of that type.
An Apple
object is a Fruit
(or Object
) object, so an Apple
object can be added into a List
of Fruit
(or Object
).
Why fruits.get(0).hello()
fails?
The actual type of the elements in the fruits
List is undetermined when declare the method.
When call a method on an element, the compiler needs to make sure the unknown type has the method.
? super Fruit
in the type argument declaration sets bounds for the unknown type, one bound (“lower”) is type Fruit
, the other bound (“upper”) is type Object
To safely invoke methods on the unknown type, only methods of the most upper type can be invoked.
Obviously, method hello
is not from the most upper type (i.e. Object
).
If passing a List<Object>
as the parameter, then obviously the elements in fruits
would not have hello()
method. Therefore the compiler raises the error.
”? extends Fruit”
void func(List<? extends Fruit> fruits){
// Error: add(capture<? extends Fruit>) in List cannot be applied to add(Fruit)
fruits.add(new Fruit());
fruits.add(new Apple()); // similar error
fruits.add(new Banana()); // similar error
fruits.get(0).hello(); // ok
}
Similar, declaring the parameter as List<? extends Fruit>
means fruits
is an unknown type,
and can be List<Fruit>
, List<Apple>
or List<Banana>
when the method is invoked.
Why fruits.get(0).hello()
is ok in this case?
Similar, ? extends Fruit
sets bounds for the unknown type, the upper bound is determined as the type Fruit
.
The hello()
method is from the most upper type, so the compiler knows it’s safe to call it within the func
body.
Why statements like fruits.add(new Fruit())
fails?
The type of fruits
is unknown, it doesn’t mean the type of fruits
is “dynamic”.
Being “dynamic” means the type of fruits
can be List<Fruit>
at some point, then be List<Apple>
at some other point.
It’s not possible in Java, since Java is a static language.
The two types, List<Fruit>
and List<Apple>
, have nothing to do with each other
(List<Apple>
is not a subtype of List<Fruit>
).
The type of the fruits
parameter is determined on the invocation of func
.
For a specific invocation, the type of fruits
is determined and static.
For example, with an invocation like func(new ArrayList<Apple>())
, the statement fruits.add(new Fruit())
would raise compiler error
(since a Fruit
cannot be add()
ed into a List<Apple>
).
To ensure all the possible invocation of func
works, the compiler just can’t allow the statements like fruits.add(new Fruit())
appear in the method body.
What is “capture” (capture<? extends Fruit>
)?
"ssh" to a Windows Host
Every now and then, we may want to connect to a Windows host via ssh
.
Maybe it’s due to the frustration on connecting a Windows host via Remote Desktop Connection over a slow network.
Below is a way to ssh
(not really, actually) to a Windows host.
Step 0, install a Linux virtual machine in the Windows host
I use Oracle Virtual Box, and install a Ubuntu guest.
Step 1, share folders between the Windows host and the guest
Share the working folders between the Windows and the Ubuntu, so that these folders are accessible within the guest. Just google “virtualbox share folder”.
Step 2, configure the guest’s network settings
Choose NAT as the adapter type.
In the “Advanced” settings, add a port forwarding rule.
Choose an available port in the Windows host for “Host Port”.
Set “Guest Port” to 22
, which is the default port sshd listening on in the Ubuntu guest.
To find an available port in the Windows host, run netsh firewall show state
in the cmd.exe.
In my corporate network, for example, port 443 is allowed.
Step 3, make a connection
Open a ssh client in another machine, type the IP (or host name) of the Windows host as the hostname,
use the “Host Port” as the port, 443
in my case, for the ssh connection.
# my_user is the username of the Ubuntu guest VM
# 1.2.3.4 is the IP of the Windows host
# 443 is the opened port in the WIndows host
ssh my_user@1.2.3.4 -p 443
# use the hostname of the Windows machine is also ok
ssh my_user@my_win_desktop.my_corp.com -p 443
After the connection is up, we’re in the Ubuntu VM. Change the directory to one of the shared folders set up in step 1, and do the work. Much better user experience than Windows Remote Desktop when the network is slow.
Other ways
One way is to set the adapter as “Bridged Adapter” for the Ubuntu guest, so that the guest get an IP from the corporate network. Configure the port of sshd to a port the corporate network allows to connect from outside. One inconvenience is that sometimes the corporate network may assign a new IP for the guest after reboot.
Another way is to install the OpenSSH Server directly in the Windows host.