Writing OpenAPI Specification
Visual Studio Code extension
I edit OpenAPI specification files in Visual Studio Code with extensions OpenAPI (Swagger) Editor and Swagger Viewer. With these extension, Visual Studio Code can edit, validate, preview OpenAPI specification files and try out API in the preview pages.
If you like these extensions, you can add them as workspace recommendations.
$ tree .vscode/
.vscode/
└── extensions.json
$ cat .vscode/extensions.json
{
// See https://go.microsoft.com/fwlink/?LinkId=827846 to learn about workspace recommendations.
// Extension identifier format: ${publisher}.${name}. Example: vscode.csharp
// List of extensions which should be recommended for users of this workspace.
"recommendations": [
"Arjun.swagger-viewer",
"42Crunch.vscode-openapi"
],
// List of extensions recommended by VS Code that should not be recommended for users of this workspace.
"unwantedRecommendations": [
]
}
The .vscode
directory can be checked into the VCS to share with others.
Using $ref
The $ref
can be mixed with “in-place” definitions.
parameters:
# Define a parameter in place
- in: path
name: id
required: true
description: The primary key.
schema:
type: integer
example: 21
# Local Reference, # means go to the root of the current document
- $ref: '#/components/parameters/ParaFoo'
# Remote Reference
- $ref: 'parameters.yaml#/components/parameters/ParaBar'
URL Reference is also supported. See more at the Using $ref documentation.
only() vs. times(1) in Mockito
Use times(1)
to verify a method is called exactly once.
verify(orderService, times(1)).getOrder(anyLong());
verify(orderService, times(1)).createOrder(any(CreateOrderRequest.class));
The above code verifies that both getOrder
and createOrder
are called exactly once on orderService
.
If the times(1)
in above example is replaced by only()
, the test would fail.
As per Mockito Javadoc of only()
,
Allows checking if given method was the only one invoked
verify(mock, only()).someMethod();
is same as
verify(mock).someMethod();
verifyNoMoreInteractions(mock);
i.e.
verify(mock, times(1)).someMethod();
verifyNoMoreInteractions(mock);
Therefore,
verify(orderService, only()).getOrder(anyLong());
verify(orderService, only()).createOrder(any(CreateOrderRequest.class));
is equivalent to
verify(orderService, times(1)).getOrder(anyLong()); // L1
verifyNoMoreInteractions(orderService); // L2
verify(orderService, times(1)).createOrder(any(CreateOrderRequest.class)); // L3
verifyNoMoreInteractions(orderService); // L4
It would fail at line “L2”.
Run GitLab CI Pipeline on Protected Branches by a Member of Developer Role
To let project members of the “Developer” role be able to run GitLab CI pipeline on protected branches, go to “Settings -> Repository -> Protected branches”, in “Allowed to merge” dropbox, add “Developers + Maintainers”.
See discussion on this GitLab issue for details.
Fix the “requested access to the resource is denied” issue
If the CI job needs to pull Docker images from registry of a GitLab project, the user who run the CI pipeline must have necessary permission to this GitLab project.
As per this GitLab document, the user should at least have the “Developer” role to pull Docker images from GitLab projects.
Action | Guest, Reporter | Developer |
---|---|---|
Pull container images from current projects | ✓ | |
Pull container images from public projects | ✓ | |
Pull container images from internal projects | ✓ |
If the user has a “Guest” or “Reporter” role in a project X, the CI job of project Y needs to pull images from the registry of project X, when that user run the CI job of project Y, the job would fail. Error message in the failed CI job would be like
pull access denied for registry.gitlab.example.com/group/project-x,
repository does not exist or may require 'docker login':
denied: requested access to the resource is denied
Why @Mock Annotated Object Is Null
Mockito provides annotations like @Mock
, @Spy
, @Captor
to make code simpler.
Since most of our code is just copied from Google search and Stack Overflow, we
may use these Mockito annotations like below, after quick reading of Google search
results like this article.
@RunWith(MockitoJUnitRunner.class)
class OrderTest {
@Captor
ArgumentCaptor<Offer> offerCaptor;
@Test
void test() {
// use the `offerCaptor` object ...
}
However, when run, the test throws NullPointerException due to the offerCaptor
is null
, even though it’s annotated by @Captor
.
(Spending hours of Googling and debugging.)
The root cause of NPE is that the test is using JUnit 5, but @RunWith
is a JUnit
4 annotation.
@RunWith no longer exists; superseded by @ExtendWith.
Therefore, @RunWith(MockitoJUnitRunner.class)
is totally ignored by JUnit 5, no
setup happens for objects annotated by @Captor
.
To support JUnit 5 @ExtendWith
, Mockito provides a MockitoExtension class.
This extension is the JUnit Jupiter equivalent of our JUnit4 MockitoJUnitRunner.
Below code has no NPE, for test cases using JUnit 5 and Mockito annotations.
@ExtendWith(MockitoExtension.class)
class OrderTest {
@Captor
ArgumentCaptor<Offer> offerCaptor;
Some Thoughts
It would be better if JUnit 5 warns us if it finds the obsolete JUnit 4 @RunWith
,
instead of failing silently.
Another solution is that, if a project is going to use JUnit 5, just exclude the JUnit 4
from the dependencies of the project. So using @RunWith
would be a compile error.
Fix a Helm "Secret Too Long" Bug
When try to upgrade a Helm chart, it fails with an error like below.
Error: UPGRADE FAILED: create: failed to create:
Secret "sh.helm.release..." is invalid:
data: Too long: must have at most 1048576 bytes
Helm by default uses Kubernetes Secrets to store release information. For what is “release information”, basically it includes the source files of the chart.
The release information includes the contents of charts and values files
See Storage backends for more details.
In my case, certificates (after encryption) are stored in the chart files, each of which is like hundreds KB big. And what’s more, each certificates has several variants for each deployment environments, like alpha, beta and production. All these files are in one helm chart. When to bring in a new certificate into the chart, the chart becomes too big, so the above error happens.
To solve this error for my chart, a simple solution to create a script to dynamically “helm ignore”
files which are not needed for a Helm release for a particular deployment environment.
For example, before do Helm release for the alpha environment, (using CI scripts to)
add files which are for production and beta into the .helmignore
file to exclude
these files from the Helm chart.
HELM_ENVS=(alpha beta prod)
for HELM_ENV in ${HELM_ENVS[@]}; do
if [ "$HELM_ENV" != "$DEPLOY_ENV" ]; then
echo "helm ignore this folder in chart: secrets/$HELM_ENV/"
echo "secrets/$APL_ENV/" >> .helmignore
fi
done
Helm also has a beta feature called “SQL storage backend” to store release information in a database, for a big chart.
Using such a storage backend is particularly useful if your release information weighs more than 1MB
CompletableFuture: join() vs. get()
What’s the difference between join()
and get()
of the CompletableFuture
class?
Both of them wait for the future completion.
The only difference is what kind of exceptions these two methods throw.
The get()
, originated in the Future
interface, throws “checked exceptions”.
While join()
throws “unchecked exceptions”.
In old days, like Java 1.5 era when Future.get()
was designed, checked exceptions might be preferred.
Nowadays, unchecked exceptions are more preferred, therefore join()
is preferred.
As per Javadoc of CompletableFuture,
To simplify usage in most contexts, this class also defines methods join() and getNow(T) that instead throw the CompletionException directly in these cases.
In other words, join()
is simpler to use, in the sense of exception handling.
It’s kind of an official recommendation.
Reminder Operator Returns Negative Value in Java
The %
operator is the reminder operator in Java.
In other languages, it may be called modulo operator.
”%”, which divides one operand by another and returns the remainder as its result.
It seems a simple operator, by looking at example code like this, 10 % 7 = 3
.
However, the %
operator may return a negative value.
assertEquals(-1, -9 % 2, "return negative value here");
assertEquals(1, 9 % -2, "return positive value here");
It’s because Java’s reminder implementation uses “truncated division”. See details in this Wiki page.
The “truncation (integer part)” of -9 / 2
, i.e. -4.5
, is -4
. So,
-9 % 2 = -9 - 2 * (-4) = -9 - (-8) = -1
For the 9 % -2
case, the “truncation” is also -4
. However,
9 % -2 = 9 - (-2) * (-4) = 9 - 8 = 1
A Problem
Sometime, this %
returning negative behavior may cause problems.
var batches = 10;
// ... some RxJava code
.groupBy(input -> Objects.hash(input) % batches)
//...
For example, in above code you may want to split your work into 10 batches.
However, because the Objects.hash()
may return negative hash code,
Objects.hash(input) % 10
has 19 possible values (integers), from -9 to 9.
So unexpectedly, your work is split int 19 batches.
A Rescue
Java 8 provides a Math.floorMod() method, which can be used in situations like above. According to its Javadoc,
If the signs of the arguments are the same, the results of floorMod and the % operator are the same.
If the signs of the arguments are different, the results differ from the % operator.
floorMod(+4, -3) == -2; and (+4 % -3) == +1
floorMod(-4, +3) == +2; and (-4 % +3) == -1
floorMod(-4, -3) == -1; and (-4 % -3) == -1
The floorMod(x, y)
is calculated using below relationship.
floorDiv(x, y) * y + floorMod(x, y) == x
And for the Math.floorDiv() method, its Javadoc says
Returns the largest (closest to positive infinity) int value that is less than or equal to the algebraic quotient.
For example, floorDiv(-4, 3) == -2, whereas (-4 / 3) == -1.
Given dividend is -4 and divisor is 3, its algebraic quotient is -1.333333.
“The largest (closest to positive infinity) int value that is less than or equal to” -1.333333
is -2, not -1 which is larger than the quotient.
Therefore, floorDiv(-4, 3) == -2
.
floorMod(-4, 3) = -4 - floorDiv(-4, 3) * 3 = -4 - (-2)*3 = 2
Compare Two OffsetDateTime of the Same Instant
For two Java OffsetDateTime objects of the same instant, but with different
timezone offset, to check whether they represent the same point in time,
the isEqual
method should be used.
var date1 = OffsetDateTime.parse("2008-12-03T11:00+01:00");
var date2 = OffsetDateTime.parse("2008-12-03T12:00+02:00");
date1.isEqual(date2); // the result is true
date1.equals(date2); // the result is false
date1.compareTo(date2); // the result < 0
In the above code, both of the OffsetDateTime objects represent “2008-12-03T10:00Z”.
However, the equals
or the compareTo
method will tell these two object are not “equal”.
The OffsetDateTime class uses a field of LocalDateTime and a field of ZoneOffset
to represent an instant. Its equals
method cares about whether all these field
of two objects are same or not. Obviously, fields of date1
are not equals with
ones of date2
.
The compareTo
has same behavior as equals
.
It is “consistent with equals”, as defined by Comparable.
As per Javadoc of OffsetDateTime, what isEqual
does is dateTime1.toInstant().equals(dateTime2.toInstant())
.
That is OffsetDateTime object are convert to Instant object first, and use
the equals
of Instant for comparison.
From the Javadoc of Instant,
An instantaneous point on the time-line.
the class stores a long representing epoch-seconds and an int representing nanosecond-of-second, … The epoch-seconds are measured from the standard Java epoch of 1970-01-01T00:00:00Z
An Instant object has no timezone info stored, only epoch of fixed UTC timezone stored, a long and a int.
Side note: what’s the different between ZonedDateTime and OffsetDateTime
The major difference is that ZonedDateTime knows Daylight saving time (DST), while OffsetDateTime does not. It’s because OffsetDateTime only stores simple offset to UTC, while ZonedDateTime stores much richer timezone info.
Git Command Examples
- Checkout a Remote Branch in Local
- Print the Short Commit SHA1 of a Git Tag
- Fix the
^M
Character Shown ingit diff
Result - Prune stale remote branches in local repository
Checkout a Remote Branch in Local
$ git checkout --track origin/master
The above command creates a local branch with the same name, i.e. master
, as the remote branch,
and let the local branch track the remote one.
“Tracking” means when run git push
, Git knows where it pushes changes into.
Some notes from git checkout --help
,
As a convenience, –track without -b implies branch creation
-t, –track
When creating a new branch, set up “upstream” configuration.
If no -b option is given, the name of the new branch will be derived from the remote-tracking branch
Print the Short Commit SHA1 of a Git Tag
# Assume the Git tag is "0.1.0"
$ git rev-list -n 1 0.1.0 --pretty=format:"%h" | tail -1
c363005
The tag “0.1.0” points to the commit c363005
.
Use %H
if the full SHA1 is needed.
(Search “placeholder” in git show --help
for the document of format:<string>
.)
Add --abbrev
option, like --abbrev=8
, if a fixed width SHA1 is needed.
Fix the ^M
Character Shown in git diff
Result
Sometimes, when run git diff
, it prints ^M
at the end of some lines.
The ^M
character represents a carriage-return character, i.e. CRLF
, the new line character in Windows.
You may see ^M
before, if you use Vim to edit some files coming for Windows/DOS.
Seeing ^M
in the git diff
result means the same line was ended with CRLF
but now
with LF
, or vice versa.
Usually, a Git repository should be configured in a way that all text files committed
into the repository end with LF
, while files checked out end the local machine specific
endings, i.e. LF
in Unix and CRLF
in Windows machines. So that ^M
would not be seen
in git diff
. To fix a repository’s configuration, add a .gitattributes
file with content
like
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto
# Declare files that will always have CRLF line endings on checkout.
*.bat text eol=crlf
“Renormalize” all the files with updated configuration.
$ git stash -u
$ git add --renormalize .
$ git status
$ git commit -m "Normalize line endings"
$ git stash pop
See this GitHub doc, Configuring Git to handle line endings, for more details.
Prune stale remote branches in local repository
As time goes by, local repository may have many remote branches, which were actually
deleted in remote repository. For example, in GitLab someone’s feature branch is usually
deleted while it’s merged by a merge request. However, these origin/feat-x
,
origin/feat-y
branches are kept in your local repository since they’re fetched.
To delete these stale remote branches in local all at once, run
$ git remote prune origin
# Or,
$ git fetch --prune
It’s said in git remote --help
,
might even prune local tags that haven’t been pushed there.
So it’s a good idea to run above commands with --dry-run
option first.
Delete one remote branch by git branch -r -d origin/feat-x
.
Mod in Java Produces Negative Numbers
Mod in Java produces negative numbers
The problem here is that in Python the % operator returns the modulus and in Java it returns the remainder. These functions give the same values for positive arguments, but the modulus always returns positive results for negative input, whereas the remainder may give negative results.
What’s the difference between “mod” and “remainder”?
-21 mod 4 is 3 because -21 + 4 x 6 is 3.
But -21 divided by 4 gives -5 with a remainder of -1.