Testing the typescript client

[wolpert]

Starting a thread for how to best test the typescript client.

[wolpert]

@bernata lets move the discussion here

[bernata]

Testing the client

The typescript client can be tested in the following ways:

1. unit tests of individual components using mocks
2. unit tests that simulate [using fakes] the environment to test across components
3. integration/end-to-end tests which run using the real java server and a headless browser.

The main developer experience goal of a unit test is you can open your IDE and run the test. No "unreasonable" setup needed. Reasonable setup includes running npm install. Unreasonable setup [for a unit test] is installing docker/podman to run the java server.

For unit tests, we prefer option 2 [simulated environment]. It runs the most code that resembles what runs in a production environment. It is simple to execute from the IDE, and from the command line. It is also easy to debug in the IDE. And most importantly it executes the seams [integration] between components. These unit tests are great for testing interesting failure paths like the connection to the server timing out, server returning an error, browser failing to create credentials, etc... And at the same time doesn't bind the tests too tightly to the implementation [i.e. we don't check if a specific product function was called in a certain way].

Option 1 is still very good and can be very useful for testing difficult-to-simulate error cases or even unique success cases.

Option 3: For this project, we treat integration and end-to-end to be the same meaning you run the real java server. You can run the client in a simple nodejs environment [closer to an integration test] or a headless browser [end-to-end test]. This test infrastructure gives the best signal for working in production, but also is hardest to setup, slowest to run, difficult/impossible to orchestrate specific codepaths, and can be cumbersome to step-through-debug across client+server.

What is a simulated enviornment?
A simulated environment for the server is an in-memory temporary http server spun up that responds to registration start/finish just like the real java server. It is flexible enough to allow a test to replace server responses/behavior to verify client behavior. But it is not resilient enough to run in production [no database persistence, no caching, no rate limiting, etc...]. The client side of a simulated environment is a nodejs runtime environment -- not a real browser.

@wolpert how's this for a start?

[wolpert]

It's a good premise, though I'm struggling with how much of a 'mock' server do we make? In other words, at what point do we just publish a swagger spec and say 'Mock this, Claude'.

Are we there yet, @bernata?

[bernata]

I think we we will implement enough of the mock server to be satisfied with the behavior of the client. And even more, to be satisifed that if we significantly refactored/re-worked the client, the tests would not need to be changed but they would still provide confidence in the behavior of the client.

[wolpert]

@bernata in general with using mock server for unit tests, and we still will run e2e tests in the CI... that's all normal. It's about how we get there.

[wolpert]

Eh... apparently Claude says the path you are doing is the best path. The mock service I was thinking wouldn't get us where it needs to be. Ask claude to write an ADR on your next PR

Carry on, wayward son. Carry on.

[bernata]

I think this is more case-by-case; so I'm ok to disagree with Claude. But in my experience, using mocks gets very product-implementation dependent and a product code change goes along with multiple test changes which shouldn't be the case IMO.

Just to clarify, what is the ADR you want to me create? Using function mocks vs using a mock server? Let me know if that's what you want. I can get that going for you after I post "finish" pr which is in-flight. Sound good?

[wolpert]

Yeah, the testing style, function mocks vs mock service.