I understand that junit is specifically for unit testing, but please read about my situation below.

I have a command line application written in Java, but I use GraalVM to compile native binaries of my app for Windows, Linux, and Mac. The issue is that sometimes the native binaries for my app will behave a little differently depending on what operating system is being used. Because of that, I want to automate the testing my command line application on all three operating systems, so that I can ensure that I easily take notice of when something is not working as expected.

To clarify, my hope is that I could use something like junit (or some other testing tool) to interact with my command line application by calling my app's native binary with some arguments, then watch for what text gets printed back out. If the text printed by my command line app matches what I expect, then the test passes.

Again, I know and understand that this is not junit's primary use case, but is this something that I can get junit to do? If so, how? If not, what other tool/framework would you recommend that I look at?

Thanks!

CodePudding user response：

Commandline Response Testing

You can include tests of calls to deployed applications via java test code Running command manually with java (using cmd.exe) https://mkyong.com/java/how-to-execute-shell-command-from-java/

I notice Picoli (command line support lib) gives some advise on testing https://picocli.info/#_black_box_and_white_box_testing

JUnit allows you to execute any Java so your testcode can use above java code examples.

Multiple OS test ENV Support

If you didnt care about the different environment it could be handled by a local unit test accessing a build output resource etc. but as you need to test on different oses then you have a dependency on the deployment and os as well so its now a system test (probably should be in TestClassIT.java naming convention for integration tests).

If you just care about OSs supported by Docker then you can use test containers to run your test locally in small docker image for each os, but if you need Windows that means you have to build and run each test on each os via VMs.

Typically people use Continuous Integration server to build and test on diff OSs. Often this is Jenkins to checkout the repo and then Jenkins slave nodes to provide the OSs, checkout the code and compile and run the commandline test. This will help provide reporting and dashboarding tools also. Other CI/SDLC tools have similar eg. TFS etc.

However - if you don't want to have another bit of infrastructure to maintain you could just have a script eg. python script to connect to machines you control to SSH in and copy local build output to the remote machine and execute and report on the test by copying back completed junit results xml file? Even cheaper have some cloud image that you just pay for while the test is running?