I tried adding the playwright tests to the workflow by modifying the corresponding file
The jobs run (in order) in the workflow are as follows:
build-backend => Sets up go and runs the command make build-server and builds the backend
ui-build => Sets up node and generates static files for the UI using following commands in order:
cd ui
npm i
cd ..
cd provider-ui
npm i
make ui-meshery-build
make ui-provider-build
test-ui-e2e => Retreives previously built static files for frontend and starts backend with the command make server. Then, runs make test-setup-ui and make test-ui to run the tests.
Now, everything is getting setup fine but the tests are failing in the github actions instance. I cleaned up the local project and used the exact same commands used in the jobs mentioned above to start a local instance, and the tests passed there.
On investigating further, I found that the auth tests on which all the other tests depend, were failing, at this line.
I eventually realized that locally, after successful login, the user gets redirected back to http://localhost:9081, whereas in the github actions instance it was getting redirected to https://mehery.layer5.io/dashboard. Thus, the test was failing.
Since we are using make commands, I’m also not sure if environment variables are causing the problem, because same variables would get used in the local instance and github actions instance as they are setup in the makefile.
(PS: I logged the redirection URL to the terminal for both my local instance and the github actions instance (in my fork repo) for investigation, and am attaching screenshots of the logs below)
No, the test simulates a email password based signin that happens after choosing provider as Meshery.
There is an environment variable named PROVIDER_TOKEN in the repository secrets, which stores my Layer5 Cloud API token, but it hasn’t been used in the test that is failing… It is used in some other tests.
Also, in the workflow file, it is set only on the test running step (here). It isn’t set while building the server and ui.
I have added the LAYER5_CLOUD_TESTING_BOT_EMAIL, LAYER5_CLOUD_TESTING_BOT_PASSWORD and PROVIDER_TOKEN variables as repository secrets in the fork, with my own credentials and Layer5 API key.
I tried setting the REMOTE_PROVIDER_URL variable before the test running step, but it still gets redirected to https://mehery.layer5.io/dashboard instead of http://localhost:9081.
Also, these variables were set by the server task of the makefile before running the server:
Why PROVIDER_BASE_URL is set to localhost:9876 instead of meshery.layer5.io?
Did you update the Makefile? As it doesn’t reflect the environment variables values that get set when you run make server.
Diff your Makefile with the one on master.
