A flexible and scalable container based Selenium Grid with video recording, live preview, basic auth & dashboard.

Zalenium

Build Status Codacy Badge Codecov GitHub release Docker Pulls Slack

Start a Selenium Grid in seconds, a grid that scales up and down dynamically with this solution based on docker-selenium to run your tests in Firefox and Chrome. If you need a different browser, Zalenium can redirect your tests to a cloud testing provider (Sauce Labs, BrowserStack, TestingBot).

Zalenium works out of the box in Docker and Kubernetes.

We improve Zalenium regularly. Please try it, and help us to improve it by reporting bugs or suggesting features through the issue tracker. Zalenium is 100% open source and it is both yours and ours, that is why we invite you to contribute to it.

This project is powered by GitHub s, support us by starring it!

Why

Thanks for open sourcing this. Our test suite run time has dropped from more than an hour to six minutes.@TKueck

We know how complicated it is to:

  • Have a stable grid to run UI tests with Selenium
  • Maintain it over time (keep up with new browser, Selenium and drivers versions)
  • Provide capabilities to cover all browsers and platforms

That is why we took this approach where docker-selenium nodes are created on demand. Your UI tests run faster in Firefox and Chrome because they are running in your own local network, on a node created from scratch and disposed after the test completes.

If you need a capability that cannot be fulfilled by docker-selenium, the test gets redirected to a cloud testing provider (Sauce Labs, BrowserStack, TestingBot).

Zalenium’s main goal is: to allow anyone to have a disposable and flexible Selenium Grid infrastructure.

Part of the idea comes from this Sauce Labs post.


What does Zalenium mean?

As you can imagine, it is the result of mixing Zalando and Selenium. As mentioned before, this project’s aim is to provide a simple way to create a grid and contribute to the Selenium community. Nevertheless, this is not an official Selenium project. We kindly ask you to create issues in this repository. If you have questions about how to get started, please join the #zalenium channel on Slack.

Try It

Get a grid up and running in a few seconds!

Prerequisites

  • Docker version >= 1.11.1 (probably works with earlier versions, not tested yet).
  • Make sure that docker info works without errors.

Run it

This is the quick start for the docker version, for kubernetes see here.

    # Pull docker-selenium
    docker pull elgalu/selenium
    
    # Pull Zalenium
    docker pull dosel/zalenium
    
    # Run it!
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      --privileged dosel/zalenium start
      
    # Point your tests to http://localhost:4444/wd/hub and run them

    # Stop
    docker stop zalenium

Try also our one line installer and starter for OSX/Linux (it will check for the latest images and ask for missing dependencies).

    curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s start
    # Point your tests to http://localhost:4444/wd/hub and run them
    curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s stop

Why --privileged? We suggest you run Zalenium as --privileged to speed up the node registration process by increasing the entropy level with Haveged. Using it is optional since it is just meant to improve its performance. For more information, check this tutorial.

  • After running the previous commands, you can check:
Features

Additional Features

  • Test status and steps directly in the video
  • Basic auth grid protection when deploying Zalenium in the cloud (AWS, GCP, …)
  • Mount volumes across containers when you need to specific files in your tests
section icon

Before starting

Make sure that docker info works without errors, and also check that you have pulled these images:

    docker pull elgalu/selenium
    docker pull dosel/zalenium

Starting Zalenium

Linux

    docker run --rm -ti --name zalenium -p 4444:4444 \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      --privileged dosel/zalenium start 

OSX

Zalenium for OSX is currently compatible with Docker 17.03.1-ce, 17.06.2-ce, and 17.09.0-ce. Nevertheless, starting with 1.13, newer CLIs can talk to older daemons. If you bump into any API compatibility issues, you can explicitly tell Zalenium which version you are using via -e DOCKER=17.06.2-ce.

    docker run --rm -ti --name zalenium -p 4444:4444 \
      -e DOCKER=17.06.2-ce \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      --privileged dosel/zalenium start

Windows

    docker run --rm -ti --name zalenium -p 4444:4444 ^
      -v /var/run/docker.sock:/var/run/docker.sock ^
      -v /c/Users/your_user_name/temp/videos:/home/seluser/videos ^
      --privileged dosel/zalenium start      

Special Cases and Customisations

(Click on the item to display its contents)

Enabling Sauce Labs (you'll need an account with them)
    export SAUCE_USERNAME=<your Sauce Labs username>
    export SAUCE_ACCESS_KEY=<your Sauce Labs access key>
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -e SAUCE_USERNAME -e SAUCE_ACCESS_KEY \
      -v /tmp/videos:/home/seluser/videos \
      -v /var/run/docker.sock:/var/run/docker.sock \
      --privileged dosel/zalenium start --sauceLabsEnabled true
    
Enabling BrowserStack (you'll need an account with them)
    export BROWSER_STACK_USER=<your BrowserStack username>
    export BROWSER_STACK_KEY=<your BrowserStack access key>
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -e BROWSER_STACK_USER -e BROWSER_STACK_KEY \
      -v /tmp/videos:/home/seluser/videos \
      -v /var/run/docker.sock:/var/run/docker.sock \
      --privileged dosel/zalenium start --browserStackEnabled true
    
Enabling TestingBot (you'll need an account with them)
    export TESTINGBOT_KEY=<your TestingBot access key>
    export TESTINGBOT_SECRET=<your TestingBot secret>
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -e TESTINGBOT_KEY -e TESTINGBOT_SECRET \
      -v /tmp/videos:/home/seluser/videos \
      -v /var/run/docker.sock:/var/run/docker.sock \
      --privileged dosel/zalenium start --testingBotEnabled true
    
Customising screen width and height, and time zone
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      --privileged dosel/zalenium start --screenWidth 1440 --screenHeight 810 --timeZone "America/Montreal"
    
Mounting volumes/folders across containers
This is a collection of folders that you can mount as volumes when starting Zalenium by prefixing the destination with /tmp/node/, and it will be mapped across all the docker-selenium containers from the root folder after stripping the /tmp/node/ prefix.

For example, mounting: -v /your/local/folder:/tmp/node/home/seluser/folder will map to /home/seluser/folder on the node.

This can be used to provide further customization to your nodes, such as adding client certificates for your browser, or mimicking prior multi-purpose folder, both shown below.
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      -v /your/local/folder/with/certStore:/tmp/node/home/seluser/.pki/nssdb \      
      -v /your/local/folderB:/tmp/node/home/seluser/folderB \      
      -v /tmp/mounted:/tmp/node/tmp/mounted \
      --privileged dosel/zalenium start
    
Please take caution in mounting system folders such as /etc, as this behavior has not been tested with such configuration.
NOTE: There are certain protected points which cannot be mounted via /tmp/node/. See PROTECTED_NODE_MOUNT_POINTS at DockerContainerClient.
Run more than one test per node
By default, Zalenium will run only one test per node/container. This behaviour can be modified by using the flag --maxTestSessions. If you setup this flag to a value higher than 1, Zalenium will run up to that given value of tests per node/container. Tuning this value for your test suites should help to reduce the overall execution time since less containers/nodes are started and stopped on demand. Here is an example:
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      --privileged dosel/zalenium start --maxTestSessions 4
    
This means that up to 4 tests will run in each node/container started by Zalenium. You could combine this parameter with --desiredContainers to get an optimal setup for your tests.

For example, if you have 20 tests that should run with 5 threads, you could start Zalenium with --desiredContainers 5 and --maxTestSessions 4. Therefore, 4 tests would be executed in each one of the 5 nodes/containers and the whole test execution should finish earlier.
Video Feature
When you start Zalenium, and you map a host folder to /home/seluser/videos, it will copy all the generated videos from the executed tests into your host mapped folder.

For example, starting Zalenium like this:
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      --privileged dosel/zalenium start 
    
will copy the generated videos to your local /tmp/videos folder. This means all videos generated from tests executed in docker-selenium containers, including the ones executed in an integrated cloud testing platform (Sauce Labs, BrowserStack, TestingBot).

The file name will be usually like this:
  • Zalenium: containerName_testName_browser_platform_timestamp.mp4

    • E.g. zalenium_myTestName_chrome_linux_20170216071201.mp4
  • Cloud Testing Platform: cloudPlatform_testName_browser_platform_timestamp.mp4

    • E.g. Sauce Labs saucelabs_myCloudTestName_safari_mac_20170216071201.mp4
    • E.g. BrowserStack browserstack_myCloudTestName_firefox_windows_20170216071201.mp4


If the test name is not set via a capability, the Selenium session ID will be used.
Accessing the host
This is the scenario where you are running some tests with Zalenium, and the SUT (system under test) is running on your host machine. Therefore, you want your tests to access your SUT.

Linux
    docker run --rm -ti --name zalenium --net=host \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      --privileged dosel/zalenium start
    # OR
    curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s start --docker-opt '--net=host' 
    
OSX

In OSX environments the --net=host flag is not supported yet. For that, we have a workaround, which is to use mac.host.local to access the host machine. So if the SUT is running on port 8080, you can do http://mac.host.local:8080 to access it.
Adding hosts to the containers
Sometimes you need to add host entries to the /etc/hosts file in order to mock dependencies, reach parts of your test infrastructure, or just to simplify your test code. Zalenium supports the --add-host flag in docker run ... and the extra_hosts option in docker-compose. Here is an example:
    # Usage:
    #   docker-compose up --force-recreate
    version: '2.1'
    
    services:
      zalenium_stg:
        image: "dosel/zalenium"
        container_name: zalenium
        hostname: zalenium
        tty: true
        volumes:
          - /tmp/videos:/home/seluser/videos
          - /var/run/docker.sock:/var/run/docker.sock
          - /usr/bin/docker:/usr/bin/docker
        ports:
          - 4444:4444
        command: >
          start --screenWidth 1930 --screenHeight 1090
                --timeZone "Asia/Tokyo"
                --videoRecordingEnabled true
                --sauceLabsEnabled false
                --browserStackEnabled false
                --testingBotEnabled false
                --startTunnel false
        extra_hosts:
          - "google.co.jp:127.0.0.1"
    
Adding proxy configuration to the containers
There might be situations where you need to add your own internal proxy configuration in case the network is very restrictive. In docker you can add the pass environment variables to overwrite that configuration in a container, e.g. http_proxy=http://myproxy.example.com:8080. Zalenium allows you to configure this values and they will be passed into the created containers. The variables are called: zalenium_http_proxy, zalenium_https_proxy, and zalenium_no_proxy. You can pass them as enviromental variables when starting Zalenium, here is an example:
    docker run --rm -ti --name zalenium -p 4444:4444 \
            -v /var/run/docker.sock:/var/run/docker.sock \
            -v /tmp/videos:/home/seluser/videos \
            -e "zalenium_http_proxy=http://myproxy.example.com:8080" \
            -e "zalenium_https_proxy=https://myproxy.example.com:8080" \
            -e "zalenium_no_proxy=172.16/12, 10.0.0.0/8, *.local, 169.254/16, 192.168.99.*, localhost, 127.0.0.1" \ 
            --privileged dosel/zalenium start 
    
Enabling basic auth
Deploying Zalenium to a cloud provider (AWS, GCP, etc...)? You can enable the basic auth feature built in Nginx to protect Zalenium when deploying it to the open internet. You can enable it in two different ways; providing a file with user(s) and password(s) or using the parameters --gridUser and --gridPassword. Here are the detailed instructions:

Providing a file with user(s) and password(s)
To create a file with that information, please follow the steps for "Creating a Password File" described in the Nginx documentation. After that, map the created file to the container when you start Zalenium, e.g.:
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      -v $(pwd)/.htpasswd:/home/seluser/.htpasswd
      --privileged dosel/zalenium start 
    
Using the --gridUser and --gridPassword parameters
    docker run --rm -ti --name zalenium -p 4444:4444 \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v /tmp/videos:/home/seluser/videos \
      --privileged dosel/zalenium start --gridUser yourUser --gridPassword yourPassword
    
Using Zalenium when the basic auth is enabled
You will need to provide the user and the password stated in the file or in the parameters at the moment of running your tests. Here is and example that shows you how to do it (the user will be yourUser and the password yourPassword).
    @Test
    public void simpleGoogleTest() throws Exception {    
        /*
           NOTE THE USE OF "yourUser" and "yourPassword" in the RemoteWebDriver url.
        */
        String URL = "http://yourUser:yourPassword@localhost:4444/wd/hub";
        DesiredCapabilities desiredCapabilities = DesiredCapabilities.chrome();
        desiredCapabilities.setCapability(CapabilityType.PLATFORM_NAME, Platform.LINUX);

        // Create a new instance of the remote web driver
        WebDriver driver = new RemoteWebDriver(new URL(URL), desiredCapabilities);

        // Maximize the window
        driver.manage().window().maximize();

        // Go to Google
        driver.get("https://www.google.com");

        // Assert that the title is the expected one
        Assert.assertEquals(driver.getTitle(), "Google", "Page title is not the expected one");

        // Close the browser
        driver.quit();
    }
    

More Configuration Parameters

Name Default Description
--desiredContainers 2 Number of nodes/containers created on startup.
--maxDockerSeleniumContainers 10 Maximum number of docker-selenium containers running at the same time.
--sauceLabsEnabled false Start Sauce Labs node or not.
--browserStackEnabled false Start BrowserStack node or not.
--testingbotEnabled false Start TestingBot node or not.
--startTunnel false When a cloud testing platform is enabled, starts the tunnel to allow local testing.

See the documentation for each provider on usage and any necessary Selenium capabilities.

The local identifier used when creating the tunnel is zalenium.
--videoRecordingEnabled true Sets if video is recorded in every test.
--screenWidth 1920 Sets the screen width.
--screenHeight 1080 Sets the screen height.
--timeZone "Europe/Berlin" Sets the time zone in the containers.
--debugEnabled false Enables LogLevel.FINE.
--seleniumImageName "elgalu/selenium" Enables overriding of the Docker selenium image to use.
--gridUser - Allows to specify a user to enable basic auth protection. --gridPassword must also be provided.
--gridPassword - Allows to specify a password to enable basic auth protection. --gridUser must also be provided.
--maxTestSessions 1 Maximum amount of tests executed per container.
--keepOnlyFailedTests false Keeps only videos of failed tests (you need to send a cookie with the test result).

One line starters

Zalenium one-liner starter

    curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash

Install and start

    curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s start

Install and start a specific version

    curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s 3.8.1a start

Cleanup

    curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s stop

Starting Zalenium with Docker Compose

Click here to display the example.
    # Usage:
    #   docker-compose up --force-recreate
    version: '2.1'
    
    services:
      #--------------#
      zalenium:
        image: "dosel/zalenium"
        container_name: zalenium
        hostname: zalenium
        tty: true
        volumes:
          - /tmp/videos:/home/seluser/videos
          - /var/run/docker.sock:/var/run/docker.sock
          - /usr/bin/docker:/usr/bin/docker
        ports:
          - 4444:4444
        command: >
          start --desiredContainers 2
                --maxDockerSeleniumContainers 8
                --screenWidth 800 --screenHeight 600
                --timeZone "Europe/Berlin"
                --videoRecordingEnabled true
                --sauceLabsEnabled false
                --browserStackEnabled false
                --testingBotEnabled false
                --startTunnel false
        environment:
          - HOST_UID
          - HOST_GID
          - SAUCE_USERNAME
          - SAUCE_ACCESS_KEY
          - BROWSER_STACK_USER
          - BROWSER_STACK_KEY
          - TESTINGBOT_KEY
          - TESTINGBOT_SECRET
    
      mock:
        image: elgalu/google_adwords_mock
        depends_on:
          - zalenium
        ports:
          - 8080:8080
        tty: true
        environment:
          - MOCK_SERVER_PORT=8080
    

`

Beware that docker-compose --abort-on-container-exit might render the video unusable, the finalization of the file cannot happen. In this case, stopping Zalenium in case of the certain conditions must be automated in another way.

section icon

Zalenium has support for Kubernetes, these instructions will give you an overview of how to get it running. If you find something that needs to be improved, please give us a hand by creating a pull request or an issue.

Kudos to @pearj for helping Zalenium work in Kubernetes.


Quick start with Minikube

You can use Minikube to deploy locally and get a first impression of Zalenium in Kubernetes. Before starting, you could follow the Hello-Minikube tutorial to get familiar with Minikube and make sure it is properly installed.

After starting Minikube locally, follow these steps:

  • (Optional) To save time, switch to the Minikube docker daemon and pull the images.
    eval $(minikube docker-env)
    docker pull elgalu/selenium
    docker pull dosel/zalenium
  • Pull the Zalenium repo to use the files from the Kubernetes folder
    git clone git@github.com:zalando/zalenium.git
  • Create the deployment in Minikube
    cd zalenium
    kubectl create -f kubernetes/
  • Go to the Minikube dashboard and check the deployment, also open the Grid Console
    # Dashboard
    minikube dashboard
    # Grid Console
    minikube service zalenium

That’s it, you can point your tests to the url obtained in the last step.


Deploying with Helm

Thanks to @gswallow for contributing to Zalenium with the Helm chart.

Helm is a tool that greatly simplifies installing apps on a Kubernetes cluster. Helm users can see an example of a Helm chart that installs Zalenium grid in the k8s/helm directory. Support can be added for different storage classes, RBAC support (and/or OpenShift).


More implementation details and deployment How Tos

Zalenium integrates with Kubernetes using the fabric8 kubernetes-client and openshift-client and the initial support was developed on OpenShift, but should be backwards compatible with vanilla Kubernetes and has been tested on Minikube.

Service Account

Zalenium uses a service account that is automatically mounted by Kubernetes, it is used to create Selenium pods and their related services.

It is a good idea to create a separate service account for specific use by Zalenium, especially when running inside OpenShift because it uses role based authentication by default, meaning that the service account will need a `ClusterRole created that has the necessary privileges to access the parts of the Kubernetes API that it needs to.

Click to see service account creation in Kubernetes
Create the Zalenium service account, this should be enough for Minikube.
    kubectl create sa zalenium
Starting from Kubernetes 1.6, there is beta RBAC support (Role Based Access Control), it is possible that this may be similar to the built-in RBAC support in OpenShift. If you want to use RBAC support in Kubernetes, you could try adapting the OpenShift instructions below.
Click to see service account creation in OpenShift
First up, create the cluster role:
    oc create -f zalenium-role.json
Then create the Zalenium service account:
    oc create sa zalenium
Then allow the Zalenium service account to run as any user, as Zalenium presently needs to run as root, which OpenShift doesn't allow by default.
    oc adm policy add-scc-to-user anyuid -z zalenium
Next add the zalenium-role you just created to the Zalenium service account.
    oc adm policy add-role-to-user zalenium-role -z zalenium
In case you get a message similar to this one:
    Error from server (NotFound): role.authorization.openshift.io "exampleview" not found
Check the namespace where you deployment is and add it to the previous command, e.g.:
    # Check namespaces
    oc get namespace
    # Execute command
    Error from server (NotFound): role.authorization.openshift.io "exampleview" not found
    oc adm policy add-role-to-user zalenium-role -z zalenium --role-namespace='your_deployment_namespace'
    

App label

Zalenium relies on there being an app="something" label that it will use to locate Services and during Pod creation. This means that you can have multiple zalenium deployments in the same kubernetes namespace that can operate independently if they have different app labels.

A good default to use would be: app=zalenium.

Overriding the Selenium Image

For performance reasons it could be a good idea to pull the selenium image, elgalu/selenium, into a local registry, especially since the image will need to be available on potentially any kubernetes node.

For more deails about overriding the Selenium image, click here
In OpenShift there is a built in registry that can automatically pull the an image from an external registry (such as docker hub) on a schedule.

This command will automatically import elgalu/selenium into the OpenShift registry at delivery/selenium:latest updating it on a schedule.
    oc tag docker.io/elgalu/selenium:latest delivery/selenium:latest --scheduled=true
This would then be available at 172.23.192.79:5000/delivery/selenium:latest in the OpenShift registry for example.

To use that image, specify --seleniumImageName 172.23.192.79:5000/delivery/selenium:latest when starting Zalenium.

Auto-mounting the shared folder

Like the Docker version of Zalenium, the Kubernetes version can automatically mount shared folders, the only catch is that when you are using persistent volumes you need to make sure that the Access Mode is set to ReadWriteMany, otherwise the selenium nodes will not be able to mount it.

Click here for more details
So for example you could create a persistent volume with these contents:
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: zalenium-shared
    spec:
      accessModes:
        - ReadWriteMany
      capacity:
        storage: 5Gi
      hostPath:
        path: /data/zalenium-shared/
And a claim like this:
    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: zalenium-shared
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 5Gi
Zalenium will scan the volumeMounts for the Zalenium container when it starts up, if it finds mounted volumes it will copy the volume mount information and the linked volume information when it creates a Selenium pod.

Managing Resources

Kubernetes has support for managing how much resources a Pod is allowed to use. Especially when using video recording it is highly recommended to specify some resource requests and/or limits otherwise users of your Kubernetes cluster may be negatively affected by the Selenium pods.

Click here for more details
There are 2 resource requests and 2 resource limits that you can set. The following table lists the possible values that you can use, however, there are no defaults, so if you don't specify anything, no resource limits or requests will be set.

Name Environment Variable Example
CPU Request ZALENIUM_KUBERNETES_CPU_REQUEST 250m (25% of a CPU core)
CPU Limit ZALENIUM_KUBERNETES_CPU_LIMIT 500m (50% of a CPU core)
Memory Request ZALENIUM_KUBERNETES_MEMORY_REQUEST 1Gi (1 Gibibyte)
Memory Limit ZALENIUM_KUBERNETES_MEMORY_LIMIT Probably best to leave empty, because Kubernetes will kill the container if it exceeds the value.

Getting Started Guidelines

Vanilla Kubernetes

Click here for more details
Create the deployment:
    kubectl run zalenium \
        --image=dosel/zalenium \
        --overrides='{"spec": {"template": {"spec": {"serviceAccount": "zalenium"}}}}' \
        -l app=zalenium,role=grid \
        -- start --desiredContainers 2
Create the services
    kubectl create service nodeport zalenium-grid --tcp=4444:4444 --dry-run -o yaml \
        | kubectl label --local -f - app=zalenium --overwrite -o yaml \
        | kubectl set selector --local -f - app=zalenium,role=grid -o yaml \
        | grep -v "running in local/dry-run mode" \
        | kubectl create -f -
Then you can open the grid in minikube by running
    minikube service zalenium-grid
For videos to work you need to mount in /home/seluser/videos.

OpenShift

Click here for more details
Create the deployment:
    oc run zalenium --image=dosel/zalenium \
        --env="ZALENIUM_KUBERNETES_CPU_REQUEST=250m" \
        --env="ZALENIUM_KUBERNETES_CPU_LIMIT=500m" \
        --env="ZALENIUM_KUBERNETES_MEMORY_REQUEST=1Gi" \
        --overrides='{"spec": {"template": {"spec": {"serviceAccount": "zalenium"}}}}' \
        -l app=zalenium,role=hub --port=4444 -- \
        start --desiredContainers 2 --seleniumImageName [registry ip address]:5000/[kubernetes namespace]/selenium:latest
Create the service
    oc create -f ./zalenium-service.yaml
In the OpenShift console you should then probably create a route. Make sure you have a proper timeout set on the route. Default in OpenShift is 30s and most probably this value is to low (pod creation of new selenium nodes might take longer time).
    oc create -f ./zalenium-route.yaml

Google Container Engine (GKE)

Click here for more details

Thanks to @laszlocph for contributing this section.

This guide can be used in addition to the information provided in the sections above.

Prerequisites
  • You have to have a Google Container Engine account with billing enabled
  • And a project created on the GKE dashboard
  • The Google Cloud SDK with the gcloud tool must be present on your machine and configured to the previously created project
  • kubectl has to be installed on your machine
Follow the Quickstart for Google Container Engine to set these up.

Creating a Kubernetes cluster
    
    gcloud container clusters create zalenium
    
    ...

    Creating cluster zalenium...done.
    Created [https://container.googleapis.com/v1/projects/xxx/zones/europe-west3-c/clusters/zalenium].
    kubeconfig entry generated for zalenium.
    NAME      ZONE            MASTER_VERSION  MASTER_IP      MACHINE_TYPE   NODE_VERSION  NUM_NODES  STATUS
    zalenium  europe-west3-c  1.6.9           aaa.bb.xxx.yy  n1-standard-1  1.6.9         3          RUNNING
    
    
Then activate the kubeconfig profile with
    
    gcloud container clusters get-credentials zalenium
    
    ...
    
    Fetching cluster endpoint and auth data.
    kubeconfig entry generated for zalenium.
    
Verify the kubectl config with kubectl get pods --all-namespaces command.

Zalenium Plumbing

Zalenium uses a Kubernetes ServiceAccount to create pods on-demand. As explained in the section above, we have to create the ServiceAccount, and we have to grant the required permissions to that account. To be able to create the roles and the necessary bindings the GKE setup has a special step, we have to make our users a cluster-admin.
   
    kubectl create clusterrolebinding <Arbitrary name for the binding, use your nickname> \
        --clusterrole=cluster-admin --user=<your google cloud login email>
Then create the necessary constructs. it also creates a Namespaces, called zalenium. You can find the plumbing.yaml file here.
    
    kubectl apply -f plumbing.yaml
For the video files, a PersistentVolume has to be created also. The pv.yaml file can be found here.
    kubectl apply -f pv.yaml
Change the kubectl context to "zalenium".
    kubectl config set-context $(kubectl config current-context) --namespace=zalenium
Launch Zalenium

Find the zalenium.yaml file here.
    kubectl apply -f zalenium.yaml
Then watch as the pods are created with kubectl get pods.
    ➜  yaml git:(kubernetes) ✗ kubectl get pods
    NAME                        READY     STATUS    RESTARTS   AGE
    zalenium-2238551656-c0w17   1/1       Running   0          4m
    zalenium-40000-17d5v        1/1       Running   0          3m
    zalenium-40001-xnqdr        1/1       Running   0          3m
You can also follow the logs with kubectl logs -f zalenium-2238551656-c0w17.

Accessing Zalenium

NodePort

Kubernetes provides multiple ways to route external traffic to the deployed services. NodePort being the most simple one and by default that is enabled in the zalenium.yaml file.

NodePort is picking a random port in the default port range (30000-32767) and makes sure that if a request is hitting that port on any of the cluster nodes, it gets routed to the deployed pod.
    $ kubectl get svc
    NAME                   CLUSTER-IP      EXTERNAL-IP   PORT(S)           AGE
    zalenium               10.43.251.95    <nodes>       4444:30714/TCP    4m
    zalenium-40000-058z8   10.43.247.200   <nodes>       50000:30862/TCP   50s
    zalenium-40001-853mq   10.43.244.122   <nodes>       50001:30152/TCP   46s
The above console output lists all services in the zalenium namespace and you can see that the hub is exposed on port 30714, and the two browser nodes on 30862 and 30152.

To access the service first you have to locate the IP address of one of the cluster nodes. The GKE cluster is built on standard Google Cloud VMs, so to find a node you have to go to the GCloud dashboard and copy an IP a node address.

Google Cloud VMs

In addition to that, you have to open the GCloud firewall too. To keep the rules flexible, but somewhat tight, the example bellow opens the firewall from a source range of IPs to all of the NodePort variations. Adjust it to your needs.
    gcloud compute firewall-rules create zalenium \
        --allow tcp:30000-32767 --source-ranges=83.94.yyy.xx/32
Zalenium is accessible on the http://35.198.142.117:30714/grid/console address in the example.

The dashboard on http://35.198.142.117:30714/dashboard/ and the "live" page on http://35.198.142.117:30714/grid/admin/live
Troubleshooting
In any case you would like to recreate the service the following one liners can assist you:

To delete the PersistentVolume and all Zalenium deployments.
    kubectl delete -f pv.yaml && kubectl delete -f zalenium.yaml
Then to recreate them
    kubectl apply -f pv.yaml && kubectl apply -f zalenium.yaml

Example Zalenium Pod Example

Click here for more details
This is an example of a working zalenium pod with all the relevant mounts attached.

    apiVersion: v1
    kind: Pod
    metadata:
      name: zalenium-test-15-lsg0v
      generateName: zalenium-test-15-
      labels:
        app: zalenium-test
    spec:
      volumes:
        - name: zalenium-videos
          persistentVolumeClaim:
            claimName: zalenium-test-videos
        - name: zalenium-shared
          persistentVolumeClaim:
            claimName: zalenium-shared
      containers:
        - name: zalenium
          image: >-
            172.23.192.79:5000/delivery/zalenium@sha256:f9ac5f4d1dc78811b7b589f0cb16fd198c9c7e562eb149b8c6e60b0686bf150f
          args:
            - start
            - '--desiredContainers'
            - '2'
            - '--screenWidth'
            - '1440'
            - '--screenHeight'
            - '810'
            - '--timeZone'
            - Australia/Canberra
            - '--seleniumImageName'
            - '172.23.192.79:5000/delivery/selenium:latest'
          ports:
            - containerPort: 4444
              protocol: TCP
          env:
            - name: ZALENIUM_KUBERNETES_CPU_REQUEST
              value: 250m
            - name: ZALENIUM_KUBERNETES_CPU_LIMIT
              value: 500m
            - name: ZALENIUM_KUBERNETES_MEMORY_REQUEST
              value: 1Gi
          resources: {}
          volumeMounts:
            - name: zalenium-videos
              mountPath: /home/seluser/videos
            - name: zalenium-shared
              mountPath: /tmp/mounted
          terminationMessagePath: /dev/termination-log
          imagePullPolicy: Always
      restartPolicy: Always
      terminationGracePeriodSeconds: 120
      dnsPolicy: ClusterFirst
      nodeSelector:
        purpose: work
      serviceAccountName: zalenium
      serviceAccount: zalenium
Usage

Live Preview

Displaying the live preview

Showing the test name on the live preview

Having a name capability with the test name will display it in the live preview. See test name for more information.

Filtering tests by build name

If more than one person is using the same instance of Zalenium, with a build capability in your tests, the live preview can be filtered to show only the tests that belong to a specific build. Pass ?build=myTestBuild at the end of the url. E.g. http://localhost:4444/grid/admin/live?build=myTestBuild

See more details at build name.


Dashboard

  • Go to http://localhost:4444/grid/dashboard

    • You can replace localhost for the IP/machine name where Zalenium is running
  • Check all the recorded videos and aggregated logs after your tests completed
  • Click on Cleanup to remove all videos and logs from the local drive and the dashboard

    • Also cleanup the dashboard via http://localhost:4444/dashboard/cleanup?action=doCleanupAll

Test Configuration Options

Test name

Click for details.
Adding a name capability with the test name will do two things; it will be displayed in the live preview to help you identify where your test is running, and the video file will also use it in the file name. Example code in Java for the capability:
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability(CapabilityType.BROWSER_NAME, BrowserType.FIREFOX);
    desiredCapabilities.setCapability(CapabilityType.PLATFORM_NAME, Platform.LINUX);
    desiredCapabilities.setCapability("name", "myTestName");

Build Name

Click for details.
Useful to filter the live preview and only display a group of tests belonging to the same build. Example code in Java for the capability:
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability(CapabilityType.BROWSER_NAME, BrowserType.CHROME);
    desiredCapabilities.setCapability(CapabilityType.PLATFORM_NAME, Platform.LINUX);
    desiredCapabilities.setCapability("build", "myTestBuild");

Idle Timeout

Click for details.
By default, Zalenium allows a test to be idle up to 90 seconds. After that elapsed time, the session will be terminated, the node will be shutdown and the recorded video will be saved (if video recording is enabled). This prevents a test to run indefinitely after something went wrong. If you need to have a longer idle timeout, just set an idleTimeout capability in your test. Example code in Java for the capability (it sets the idleTimeout to 150 seconds):
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability(CapabilityType.BROWSER_NAME, BrowserType.FIREFOX);
    desiredCapabilities.setCapability(CapabilityType.PLATFORM_NAME, Platform.LINUX);
    desiredCapabilities.setCapability("idleTimeout", 150);

Screen Resolution

Click for details.
You can pass a custom screen resolution for your test, just include a screenResolution with the desired value. E.g. screenResolution=1280x1024. Also supported for the same purpose resolution and screen-resolution. Example code in Java for the capability screenResolution
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability(CapabilityType.BROWSER_NAME, BrowserType.FIREFOX);
    desiredCapabilities.setCapability(CapabilityType.PLATFORM_NAME, Platform.LINUX);
    desiredCapabilities.setCapability("screenResolution", "1280x720");

Disable Video Recording

Click for details.
It is possible to disable video recording (enabled by default) via test capabilities. Add a recordVideo=false capability and no video will be recorded. Example code in Java for the capability recordVideo
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability(CapabilityType.BROWSER_NAME, BrowserType.FIREFOX);
    desiredCapabilities.setCapability(CapabilityType.PLATFORM_NAME, Platform.LINUX);
    desiredCapabilities.setCapability("recordVideo", false);

Time Zone

Click for details.
Run your test in a different time zone from the default one Europe/Berlin, just pass a capability tz with the desired value. E.g. tz=America/MontrealExample code in Java for the capability tz.
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability(CapabilityType.BROWSER_NAME, BrowserType.FIREFOX);
    desiredCapabilities.setCapability(CapabilityType.PLATFORM_NAME, Platform.LINUX);
    desiredCapabilities.setCapability("tz", "America/Montreal");

Marking the test as passed or failed

Click for details.
By default, tests in Zalenium are marked in the dashboard either as COMPLETED (session finishes normally) or TIMEOUT (session was ended due to inactivity). You can mark the test as passed or failed based on the assertions you do on your side with your test framework, add a cookie from with the name zaleniumTestPassed with a value of true (if the test passes) or false (if the test fails). This could be done in the after method where you already know if the test passed or failed. Here is an example in Java:
    Cookie cookie = new Cookie("zaleniumTestPassed", "true");
    webDriver.manage().addCookie(cookie);

Referencing test steps in the recorded video

Click for details.
It is possible to reference your tests steps in the recorded video by passing their description to Zalenium via a cookie. For example, your test could go to the home page, search and add an article to the basket, go to the checkout, and pay. All this steps can be referenced in the video for a more simple debugging. You can pass the steps via messages with a cookie named zaleniumMessage. Here is an example in Java:
    Cookie cookie = new Cookie("zaleniumMessage", "Go to home page");
    webDriver.manage().addCookie(cookie);
    webDriver.get("http://www.homepage.com");
    
    cookie = new Cookie("zaleniumMessage", "Search and add article to the basket");
    webDriver.manage().addCookie(cookie);
    /*
        Code performing WebDriver actions to search and add article to the basket.
     */
    
    cookie = new Cookie("zaleniumMessage", "Go to the checkout");
    webDriver.manage().addCookie(cookie);
    /*
        Code performing WebDriver actions to go to the checkout.
     */
    
    cookie = new Cookie("zaleniumMessage", "Pay");
    webDriver.manage().addCookie(cookie);
    /*
        Code performing WebDriver actions to pay.
     */
     

Set browser language (works only with Chrome)

Click for details.
You can set the browser language when using Google Chrome, just pass the ChromeOptions variable with the language argument. Example code in Java :
    DesiredCapabilities desiredCapabilities = DesiredCapabilities.chrome();
    ChromeOptions options = new ChromeOptions();
    options.addArguments("lang=en_GB");
    desiredCapabilities.setCapability(ChromeOptions.CAPABILITY, options);

Waiting for Zalenium to be ready

A simple way to check if Zalenium is ready to receive test requests is to use the built-in status url that Selenium Grid already provides: http://localhost:4444/wd/hub/status

If the endpoint returns an HTTP code 200 and the value for the key ready is true, it means that Zalenium is ready to receive tests requests. Here is an example how the JSON payload looks like:

`

    {
      "status": 0,
      "value": {
        "ready": true,
        "message": "Hub has capacity",
        "build": {
          "revision": "6e95a6684b",
          "time": "2017-12-01T19:05:32.194Z",
          "version": "3.8.1"
        },
        "os": {
          "arch": "amd64",
          "name": "Linux",
          "version": "4.9.49-moby"
        },
        "java": {
          "version": "1.8.0_151"
        }
      }
    }

You could grep the value using jq in a bash function like this:

    curl -sSL http://localhost:4444/wd/hub/status | jq .value.ready | grep true
How

How it works

Zalenium works conceptually in a simple way:

  1. A Selenium Hub starts and listens on port 4444.
  2. One custom node for docker-selenium
  3. If a cloud testing integration is enabled, a cloud proxy node to support a cloud provider (Sauce Labs, BrowserStack, TestingBot) will register to the grid.
  4. A test request is received by the hub and the requested capabilities are verified against each one of the nodes.
  5. If docker-selenium can fulfill the requested capabilities, a docker container is created on the run, and the test request is sent back to the hub while the new node registers.
  6. The hub acknowledges the new node and routes the the test request with to it.
  7. The test is executed and the container is disposed after test completion.
  8. If docker-selenium cannot fulfill the requested capabilities, it will processed by one of the enabled cloud testing platforms.

Project Versioning

  • To make life easy for people who want to use Zalenium, we are now using as a version number the Selenium version being supported.
  • The major-minor version combined with the patch level will indicate the Selenium version being supported. E.g.

    • When a release is 3.8.1a, it supports Selenium 3.8.1
    • The badge above shows the latest image version
    • Alias for the latest images, dosel/zalenium:latest
Contributing

Contributions

Thank you for your interest in making this project even better and more awesome. Your contributions are highly welcomed.

If you need help, please open a GitHub Issue in this project. If you work at Zalando reach out to us at team-tip.

Report a bug

Reporting bugs is one of the best ways to contribute. Before creating a bug report, please check that an issue reporting the same problem does not already exist. If there is an such an issue, you may add your information as a comment.

To report a new bug, open an issue that summarizes the bug and set the label to “bug”.

If you want to provide a fix along with your bug report: That is great! In this case please send us a pull request as described in section Contribute Code.

Suggest a Feature

To request a new feature, open an issue and summarize the desired functionality and its use case. Set the issue label to “feature”.

Contribute Code

This is a rough outline of what the workflow for code contributions looks like:

  • Check the list of open issues. Either assign an existing issue to yourself, or create a new one that you would like work on and discuss your ideas and use cases.
  • Fork the repository
  • Create a topic branch from where you want to base your work. This is usually master.
  • Make commits of logical units.
  • Write good commit messages (see below).
  • Push your changes to a topic branch in your fork of the repository.
  • Submit a pull request
  • Your pull request must receive a from two maintainers

Thanks for your contributions!

Commit messages

Your commit messages ideally can answer two questions: what changed and why. The subject line should feature the “what” and the body of the commit should describe the “why”.

When creating a pull request, its comment should reference the corresponding issue id.

Have fun and enjoy hacking!


Code of Conduct

We have adopted the Contributor Covenant as the code of conduct for this project:

http://contributor-covenant.org/version/1/4/


Building and Testing

If you want to verify your changes locally with the existing tests (please double check that the Docker daemon is running):

  • Unit tests
    mvn clean test
  • Building the image
    mvn clean package
    cd target
    docker build -t zalenium:YOUR_TAG .
  • Running the image you just built
    docker run --rm -ti --name zalenium -p 4444:4444 \
        -v /var/run/docker.sock:/var/run/docker.sock \
        -v /tmp/videos:/home/seluser/videos \
        --privileged zalenium:YOUR_TAG start
  • Running the integration tests with Sauce Labs or BrowserStack or TestingBot. You will need an account on any of those providers to run them (they have free plans).
    ./run_integration_tests.sh sauceLabs|browserStack|testingBot