Grails 3.1.9 Functional Test Port
With Grails 3.1.9 and above we now get a default port for Functional tests that is randomly assigned. This is a great feature that was added in with this commit.
While this is a nice addition our tests which were not using Geb were left broken this is due to the fact we now need to know what port was selected for a given run of the tests. So our old test:
Was broken since the app no longer started on 8080, but we can fix that with a new property that the @Integration
annotation adds which is serverPort
so the test becomes:
If you are wondering how serverPort
is actually set you can see it happening inside IntegrationTestMixinTransformation. It will add a property to the test class with the port.
Grails 3 Metrics and Path Variables
Grails 3 is built on top of SpringBoot on of the key components of SpringBoot is Actuator. Actuator sets up Metrics and healthchecks all the things modern services are expected to just have. Grails 3 ships with the /metrics
endpoint turned on meaning every request gets tracked by default the endpoint will be available to all logged in users assuming you are using some security.
Most of the time this wouldn’t be an issue and would be very nice to have those metrics the problem comes to the fact that currently all requests with Path Variables are treated as different endpoints. So requests to /items/$id
of /items/123
and /items/333
will end up listed as two different metrics with the value of the path variable for everyone to see. This exposes more data than intended and doesn’t provide really useful metrics. Its due to an issue in Actuator GH Issue 5875. So until this is fixed I recommend turning off metrics collection all together for Grails 3.
You can do that with the following config in your application.yml
I just keep the metrics endpoint off since the health-checks are useful and something we depend on others may want to just shut of Actuator fully which can be done with:
Thanks to Ryan Vanderwerf for help on this issue.
RxJava and Ratpack Testing Gotcha
Ratpack tends to stay out of the way when building out functionality so there are times you may make classes that use RxJava and Guice without anything from Ratpack. But there is one caveat during testing, if you are using RxJava in a unit test without any Ratpack integration that will work just fine. As soon as there is another test in the same execution that uses Ratpacks integration (ratpack.dependency("rx")
) with RxJava such as a functional test you will start seeing the following error in the standard out:
Many of your functional tests will start failing with very unpredictable results. What is happening is any use of RxJava that runs before the Ratpacks integration will set up an execution hook meaning the strong guarantees about execution Ratpack provides will not be available leading to unpredictable behavior. This is an easy fix for the test, we just need Ratpack to setup the hook in these other tests we can simply add the following to a Spock spec:
The same initialize can be used in your favorite testing tools, it just needs to happen before the tests are run once and thats what setupSpec
gives us in Spock.
Grails 3 and JaCoCo
Now that we have Gradle as our build system we have a whole range of plugins we can use directly in Gradle. For code coverage I am using the JaCoCo plugin. To use it with Grails we just apply the plugin to the build. By default you will get a HTML report, in the build/report/jacoco
directory.
But Grails 3 has it’s test phases split up, out of the box you will have test
and integrationTest
. Starting with the JaCoCo gradle plugin, you will get coverage for just your test
phase.
We can easily fix this by letting JaCoCo know about the phases we want reported on so we just adjust the config for the jacocoTestReport task.
Vertx3 and Gradle Application Plugin
Vertx 3 is a great step forward you can now work with Vertx without having to install a command line tool to run it. Now you can use standard Maven or Gradle to run your application.
With Gradle the samples given all are based on the use of the great Shadow Plugin. While this is good for deployments and most local development I missed a few of the features of the application plugin.
Set up your build.gradle
with the following:
In this case we are deploying Vertx with verticles. So we set the main verticle which does our setup. I define that as variable since we will need to use it in two places once as an argument to the Vertx starter and again in the Jar manifest. You will note the main verticle includes the type in our case it is Groovy but if you are doing JavaScript you would change it there.
Now we can take advantage of the application plugin allowing us to use the simple gradle run
instead of depending on the Shadow plugin it is also faster then rebuilding the whole fat jar.
But the main reason I wanted to use the application plugin was the ability to debug easily.
That command will allow us to attach a debugger to 5005 default. With these changes Vertx 3 development fits in along side my other Grails 3 and Java 8 projects easily.
Grails 3 and Spring Java Config
The Spring Java config is a great way to work with Spring configuration. It feels similar to Google Guice, which I personally enjoy. Mr. Haki did a great write up on using it with Grails 2.4.
While working with Grails 3 there are a few areas that don’t have plugins yet but do have Spring modules we can leverage. So I started out with the same path given for Grails 2.X.
I found that while the beans were all wired correctly and @Autowired
all worked correctly, but any mappings with @RequestMapping
where always returning a 404. Everything looked wired correctly but Grails wasn’t letting any requests through.
So I switched over to using @ComponentScan
then the requests were properly mapped and everything started working.
Using ComponentScan with Grails 3 is very easy:
This will scan the package of example.config
for any Spring components so your Spring Java configs can all live in this package and easily get picked up.
This post is the blog form of this SO Question.
SpringSecurity Impersonate Users Custom Roles
The Grails SpringSecurity plugin has the ability to allow user impersonation which is a really great tool for support. But many times it makes sense to allow your support user to see different things than a user. We use a special role to achieve this behavior we can also have different roles for tiers of support.
By default even impersonation or user switching is turned off in the plugin by default. You can easily turn it on with a config flag:
Spring Security by default will add a new role of ROLE_PREVIOUS_ADMINISTRATOR for the user while impersonating the other user. This will work in most cases but for us it was hard to reason about since we had an idea of admin which was separate from our support team who would mostly be using this tool.
So for our use we wanted our own roles. The ability to do this is exposed in the SwitchUserFilter calling SwitchUserAuthorityChanger. With an implementation of the interface you can add any roles you would like.
Example authority changer:
In Grails we set our authority changer during bootstrap.
The switch user filter is already exposed as a bean so we can just get that injected and we also exposed the authority changer as a bean so we could just inject it in bootstrap.
There are many other options to think about when allowing impersonation of users, it may even make sense to remove some roles from an impersonated user. That is easily done by simply removing roles from the list retuned from the authority changer.
Cassandra Stale Snapshots
While working with Cassandra we found that sometimes snapshots get left around after repairs fail or have issues. These can cause 100’s of Gigs of space to just be wasted. This can add up quickly and cause issues. On a node with low disk space we can check for stale snapshots the following way.
If you see large or multiple directories under columnFamily/snapshots/ that may indicate there are stale snapshots that can be cleaned up. We will want to check when these snapshots where created so inside the snapshots directory of the column family run
Each of those are snapshots but you will notice 4f37dcd0-c7fa-11e4-b5ae-5f969a9b23c8 is stale, we can now clean that up with the nodetool clearsnapshot command.
That will take care of removing the stale snapshot you should see disk space recovered immediately.
You can also look in the logs for failed repairs we see something like the following:
In that case the bad snapshot would be 72e69720-b959-11e4-9b55-39152d07d3bf.
Grails Integration Testing of Complex Transactions
Integration testing complex transactions in grails can be tricky due to the default behavior of wrapping integration tests in transactions and rolling them back when complete.
The simple solution is to simply turn off transactions for integration tests, that solution will work but tends to lead to data pollution in downstream tests. Burt Beckwith has a solution in his post An Alternative Approach for Grails Integration Tests. Using this solution we can rebuild the database for each test.
In general most of the time you can use the default transactional testing behavior, therefore we want to only use this method some of the time. To achieve this effect we will modify Burt’s original solution slightly.
- Rebuild the database after each test
- Depend on the configured data source
- Update to Spock
Updated Solution
You’ll notice we extend a company.IntegrationSpec there isn’t anything special in that just a base integration spec class that can hold our shared testing code such as bootstrapping methods and security methods. Since our data bootstrap logic is set up there we can share the same initial setup across both our normal integration test and our new non-transactional integration specs.
When to Use
The main time we started to need this new testing method was when doing custom propagation of hibernate transactions.
Since this will require a new transaction we will not be able to see the inserts that have happened in the default transaction. We do many of our reads as read only and force a new transaction when we are more read heavy during an operation.
In general anytime you are working directly with transaction rollbacks or propagation it best to test those things directly without having Grails inject an extra transaction in there for you.
Memcached Tomcat Sessions and Grails App Info Plugin
The App Info Plugin for Grails is a great tool. We use it in most of our Grails deployments. One of the great features offered is the ability to view all current sessions and their details. Another tool we are using is Memcached Session Manager which allows shared sessions between our Tomcat servers, great for rolling deploys and high availability.
We found an issue while running both of these in production. Over time the number of sessions reported by App Info would grow extremely large. It turns out that having the App Info plugin track sessions for you when the sessions are shared via memcached can lead to the plugin holding onto references to sessions that have been destroyed on other severs.
So at this time we can’t use the App Info plugin session tracking with the Memcached Session Manager, since it leads to a memory leak as all those sessions have references inside the plugin and can never be garbage collected.
We can simply turn off the session tracking with the following code:
But note this config change effect will effect how a war is made so you can’t simply override this in production you will need a new war built.
Below is our memory graph before and after the change: