Visual Studio Code extension

I edit OpenAPI specification files in Visual Studio Code with extensions OpenAPI (Swagger) Editor and Swagger Viewer. With these extension, Visual Studio Code can edit, validate, preview OpenAPI specification files and try out API in the preview pages.

If you like these extensions, you can add them as workspace recommendations.

$ tree .vscode/
.vscode/
└── extensions.json

$ cat .vscode/extensions.json
{
  // See https://go.microsoft.com/fwlink/?LinkId=827846 to learn about workspace recommendations.
  // Extension identifier format: ${publisher}.${name}. Example: vscode.csharp
  // List of extensions which should be recommended for users of this workspace.
  "recommendations": [
    "Arjun.swagger-viewer",
    "42Crunch.vscode-openapi"
  ],
  // List of extensions recommended by VS Code that should not be recommended for users of this workspace.
  "unwantedRecommendations": [

  ]
}

The .vscode directory can be checked into the VCS to share with others.

Using $ref

The $ref can be mixed with “in-place” definitions.

      parameters:
        # Define a parameter in place
        - in: path
          name: id
          required: true
          description: The primary key.
          schema:
            type: integer
          example: 21
        # Local Reference, # means go to the root of the current document
        - $ref: '#/components/parameters/ParaFoo'
        # Remote Reference
        - $ref: 'parameters.yaml#/components/parameters/ParaBar'

URL Reference is also supported. See more at the Using $ref documentation.

Use times(1) to verify a method is called exactly once.

  verify(orderService, times(1)).getOrder(anyLong());
  verify(orderService, times(1)).createOrder(any(CreateOrderRequest.class));

The above code verifies that both getOrder and createOrder are called exactly once on orderService.

If the times(1) in above example is replaced by only(), the test would fail.

As per Mockito Javadoc of only(),

Allows checking if given method was the only one invoked

verify(mock, only()).someMethod(); is same as

  verify(mock).someMethod();
  verifyNoMoreInteractions(mock);

i.e.

  verify(mock, times(1)).someMethod();
  verifyNoMoreInteractions(mock);

Therefore,

  verify(orderService, only()).getOrder(anyLong());
  verify(orderService, only()).createOrder(any(CreateOrderRequest.class));

is equivalent to

  verify(orderService, times(1)).getOrder(anyLong()); // L1
  verifyNoMoreInteractions(orderService); // L2
  verify(orderService, times(1)).createOrder(any(CreateOrderRequest.class)); // L3
  verifyNoMoreInteractions(orderService); // L4

It would fail at line “L2”.

To let project members of the “Developer” role be able to run GitLab CI pipeline on protected branches, go to “Settings -> Repository -> Protected branches”, in “Allowed to merge” dropbox, add “Developers + Maintainers”.

See discussion on this GitLab issue for details.

Fix the “requested access to the resource is denied” issue

If the CI job needs to pull Docker images from registry of a GitLab project, the user who run the CI pipeline must have necessary permission to this GitLab project.

As per this GitLab document, the user should at least have the “Developer” role to pull Docker images from GitLab projects.

If the user has a “Guest” or “Reporter” role in a project X, the CI job of project Y needs to pull images from the registry of project X, when that user run the CI job of project Y, the job would fail. Error message in the failed CI job would be like

pull access denied for registry.gitlab.example.com/group/project-x, 
repository does not exist or may require 'docker login': 
denied: requested access to the resource is denied

Mockito provides annotations like @Mock, @Spy, @Captor to make code simpler. Since most of our code is just copied from Google search and Stack Overflow, we may use these Mockito annotations like below, after quick reading of Google search results like this article.

@RunWith(MockitoJUnitRunner.class)
class OrderTest {
    @Captor
    ArgumentCaptor<Offer> offerCaptor;

    @Test
    void test() {
      // use the `offerCaptor` object ...
    }

However, when run, the test throws NullPointerException due to the offerCaptor is null, even though it’s annotated by @Captor.

(Spending hours of Googling and debugging.)

The root cause of NPE is that the test is using JUnit 5, but @RunWith is a JUnit 4 annotation.

@RunWith no longer exists; superseded by @ExtendWith.

Therefore, @RunWith(MockitoJUnitRunner.class) is totally ignored by JUnit 5, no setup happens for objects annotated by @Captor.

To support JUnit 5 @ExtendWith, Mockito provides a MockitoExtension class.

This extension is the JUnit Jupiter equivalent of our JUnit4 MockitoJUnitRunner.

Below code has no NPE, for test cases using JUnit 5 and Mockito annotations.

@ExtendWith(MockitoExtension.class)
class OrderTest {
    @Captor
    ArgumentCaptor<Offer> offerCaptor;

Some Thoughts

It would be better if JUnit 5 warns us if it finds the obsolete JUnit 4 @RunWith, instead of failing silently.

Another solution is that, if a project is going to use JUnit 5, just exclude the JUnit 4 from the dependencies of the project. So using @RunWith would be a compile error.

When try to upgrade a Helm chart, it fails with an error like below.

Error: UPGRADE FAILED: create: failed to create: 
Secret "sh.helm.release..." is invalid: 
data: Too long: must have at most 1048576 bytes

Helm by default uses Kubernetes Secrets to store release information. For what is “release information”, basically it includes the source files of the chart.

The release information includes the contents of charts and values files

See Storage backends for more details.

In my case, certificates (after encryption) are stored in the chart files, each of which is like hundreds KB big. And what’s more, each certificates has several variants for each deployment environments, like alpha, beta and production. All these files are in one helm chart. When to bring in a new certificate into the chart, the chart becomes too big, so the above error happens.

To solve this error for my chart, a simple solution to create a script to dynamically “helm ignore” files which are not needed for a Helm release for a particular deployment environment. For example, before do Helm release for the alpha environment, (using CI scripts to) add files which are for production and beta into the .helmignore file to exclude these files from the Helm chart.

HELM_ENVS=(alpha beta prod)
for HELM_ENV in ${HELM_ENVS[@]}; do
  if [ "$HELM_ENV" != "$DEPLOY_ENV" ]; then
      echo "helm ignore this folder in chart: secrets/$HELM_ENV/"
      echo "secrets/$APL_ENV/" >> .helmignore
  fi
done

Helm also has a beta feature called “SQL storage backend” to store release information in a database, for a big chart.

Using such a storage backend is particularly useful if your release information weighs more than 1MB

What’s the difference between join() and get() of the CompletableFuture class?

Both of them wait for the future completion. The only difference is what kind of exceptions these two methods throw. The get(), originated in the Future interface, throws “checked exceptions”. While join() throws “unchecked exceptions”.

In old days, like Java 1.5 era when Future.get() was designed, checked exceptions might be preferred. Nowadays, unchecked exceptions are more preferred, therefore join() is preferred.

As per Javadoc of CompletableFuture,

To simplify usage in most contexts, this class also defines methods join() and getNow(T) that instead throw the CompletionException directly in these cases.

In other words, join() is simpler to use, in the sense of exception handling. It’s kind of an official recommendation.

The % operator is the reminder operator in Java. In other languages, it may be called modulo operator.

”%”, which divides one operand by another and returns the remainder as its result.

It seems a simple operator, by looking at example code like this, 10 % 7 = 3.

However, the % operator may return a negative value.

  assertEquals(-1, -9 % 2, "return negative value here");
  assertEquals(1, 9 % -2, "return positive value here");

It’s because Java’s reminder implementation uses “truncated division”. See details in this Wiki page. The “truncation (integer part)” of -9 / 2, i.e. -4.5, is -4. So,

  -9 % 2 = -9 - 2 * (-4) = -9 - (-8) = -1

For the 9 % -2 case, the “truncation” is also -4. However,

  9 % -2 = 9 - (-2) * (-4) = 9 - 8 = 1

A Problem

Sometime, this % returning negative behavior may cause problems.

  var batches = 10;
  // ... some RxJava code
  .groupBy(input -> Objects.hash(input) % batches)
  //...

For example, in above code you may want to split your work into 10 batches. However, because the Objects.hash() may return negative hash code, Objects.hash(input) % 10 has 19 possible values (integers), from -9 to 9. So unexpectedly, your work is split int 19 batches.

A Rescue

Java 8 provides a Math.floorMod() method, which can be used in situations like above. According to its Javadoc,

If the signs of the arguments are the same, the results of floorMod and the % operator are the same.

If the signs of the arguments are different, the results differ from the % operator.

floorMod(+4, -3) == -2;   and (+4 % -3) == +1
floorMod(-4, +3) == +2;   and (-4 % +3) == -1
floorMod(-4, -3) == -1;   and (-4 % -3) == -1

The floorMod(x, y) is calculated using below relationship.

floorDiv(x, y) * y + floorMod(x, y) == x

And for the Math.floorDiv() method, its Javadoc says

Returns the largest (closest to positive infinity) int value that is less than or equal to the algebraic quotient.

For example, floorDiv(-4, 3) == -2, whereas (-4 / 3) == -1.

Given dividend is -4 and divisor is 3, its algebraic quotient is -1.333333. “The largest (closest to positive infinity) int value that is less than or equal to” -1.333333 is -2, not -1 which is larger than the quotient. Therefore, floorDiv(-4, 3) == -2.

  floorMod(-4, 3) = -4 - floorDiv(-4, 3) * 3 = -4 - (-2)*3 = 2

For two Java OffsetDateTime objects of the same instant, but with different timezone offset, to check whether they represent the same point in time, the isEqual method should be used.

var date1 = OffsetDateTime.parse("2008-12-03T11:00+01:00");
var date2 = OffsetDateTime.parse("2008-12-03T12:00+02:00");
date1.isEqual(date2); // the result is true
date1.equals(date2); // the result is false
date1.compareTo(date2); // the result < 0

In the above code, both of the OffsetDateTime objects represent “2008-12-03T10:00Z”. However, the equals or the compareTo method will tell these two object are not “equal”.

The OffsetDateTime class uses a field of LocalDateTime and a field of ZoneOffset to represent an instant. Its equals method cares about whether all these field of two objects are same or not. Obviously, fields of date1 are not equals with ones of date2.

The compareTo has same behavior as equals.

It is “consistent with equals”, as defined by Comparable.

As per Javadoc of OffsetDateTime, what isEqual does is dateTime1.toInstant().equals(dateTime2.toInstant()). That is OffsetDateTime object are convert to Instant object first, and use the equals of Instant for comparison.

From the Javadoc of Instant,

An instantaneous point on the time-line.

the class stores a long representing epoch-seconds and an int representing nanosecond-of-second, … The epoch-seconds are measured from the standard Java epoch of 1970-01-01T00:00:00Z

An Instant object has no timezone info stored, only epoch of fixed UTC timezone stored, a long and a int.

Side note: what’s the different between ZonedDateTime and OffsetDateTime

The major difference is that ZonedDateTime knows Daylight saving time (DST), while OffsetDateTime does not. It’s because OffsetDateTime only stores simple offset to UTC, while ZonedDateTime stores much richer timezone info.

• Checkout a Remote Branch in Local
• Print the Short Commit SHA1 of a Git Tag
• Fix the ^M Character Shown in git diff Result
• Prune stale remote branches in local repository

Checkout a Remote Branch in Local

$ git checkout --track origin/master

The above command creates a local branch with the same name, i.e. master, as the remote branch, and let the local branch track the remote one. “Tracking” means when run git push, Git knows where it pushes changes into.

Some notes from git checkout --help,

As a convenience, –track without -b implies branch creation

-t, –track
When creating a new branch, set up “upstream” configuration.

If no -b option is given, the name of the new branch will be derived from the remote-tracking branch

Print the Short Commit SHA1 of a Git Tag

# Assume the Git tag is "0.1.0"
$ git rev-list -n 1 0.1.0 --pretty=format:"%h" | tail -1
c363005

The tag “0.1.0” points to the commit c363005. Use %H if the full SHA1 is needed.

(Search “placeholder” in git show --help for the document of format:<string>.)

Add --abbrev option, like --abbrev=8, if a fixed width SHA1 is needed.

Fix the ^M Character Shown in git diff Result

Sometimes, when run git diff, it prints ^M at the end of some lines. The ^M character represents a carriage-return character, i.e. CRLF, the new line character in Windows. You may see ^M before, if you use Vim to edit some files coming for Windows/DOS. Seeing ^M in the git diff result means the same line was ended with CRLF but now with LF, or vice versa.

Usually, a Git repository should be configured in a way that all text files committed into the repository end with LF, while files checked out end the local machine specific endings, i.e. LF in Unix and CRLF in Windows machines. So that ^M would not be seen in git diff. To fix a repository’s configuration, add a .gitattributes file with content like

# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto

# Declare files that will always have CRLF line endings on checkout.
*.bat text eol=crlf

“Renormalize” all the files with updated configuration.

$ git stash -u
$ git add --renormalize .
$ git status
$ git commit -m "Normalize line endings"
$ git stash pop

See this GitHub doc, Configuring Git to handle line endings, for more details.

Prune stale remote branches in local repository

As time goes by, local repository may have many remote branches, which were actually deleted in remote repository. For example, in GitLab someone’s feature branch is usually deleted while it’s merged by a merge request. However, these origin/feat-x, origin/feat-y branches are kept in your local repository since they’re fetched.

To delete these stale remote branches in local all at once, run

$ git remote prune origin

# Or,
$ git fetch --prune

It’s said in git remote --help,

might even prune local tags that haven’t been pushed there.

So it’s a good idea to run above commands with --dry-run option first.

Delete one remote branch by git branch -r -d origin/feat-x.

Mod in Java produces negative numbers

The problem here is that in Python the % operator returns the modulus and in Java it returns the remainder. These functions give the same values for positive arguments, but the modulus always returns positive results for negative input, whereas the remainder may give negative results.

What’s the difference between “mod” and “remainder”?

-21 mod 4 is 3 because -21 + 4 x 6 is 3.

But -21 divided by 4 gives -5 with a remainder of -1.