399 lines
16 KiB
Markdown
399 lines
16 KiB
Markdown
---
|
|
stage: Verify
|
|
group: Continuous Integration
|
|
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
|
|
type: tutorial
|
|
---
|
|
|
|
# Testing a Phoenix application with GitLab CI/CD
|
|
|
|
[Phoenix](https://www.phoenixframework.org/) is a web development framework written in [Elixir](https://elixir-lang.org), which is a
|
|
functional language designed for productivity and maintainability that runs on the
|
|
[Erlang VM](https://www.erlang.org). Erlang VM is really fast and can handle very large numbers of
|
|
simultaneous users.
|
|
|
|
That's why we're hearing so much about Phoenix today.
|
|
|
|
In this tutorial, we'll teach you how to set up [GitLab CI/CD](../../README.md) to build and test a Phoenix
|
|
application.
|
|
|
|
The tutorial assumes that you know how to create a Phoenix app, run tests locally, and how to work with Git
|
|
and the GitLab UI.
|
|
|
|
## Introduction
|
|
|
|
### What is Phoenix?
|
|
|
|
[Phoenix](https://www.phoenixframework.org/) is a web development framework written in [Elixir](https://elixir-lang.org). It's useful
|
|
for building fast, reliable, and high-performance applications, as it uses [Erlang VM](https://www.erlang.org).
|
|
|
|
Many components and concepts are similar to Ruby on Rails or Python's Django. High developer
|
|
productivity and high application performance are only a few advantages on learning how to use it.
|
|
Working on the MVC pattern, it's was designed to be modular and flexible. Easy to maintain a growing
|
|
app is a plus.
|
|
|
|
Phoenix can run in any OS where Erlang is supported:
|
|
|
|
- Ubuntu
|
|
- CentOS
|
|
- Mac OS X
|
|
- Debian
|
|
- Windows
|
|
- Fedora
|
|
- Raspberry Pi OS
|
|
|
|
Check the [Phoenix learning guide](https://hexdocs.pm/phoenix/overview.html) for more information.
|
|
|
|
### What is Elixir?
|
|
|
|
[Elixir](https://elixir-lang.org) is a dynamic, functional language created to use all the maturity of Erlang
|
|
(30 years old!) in these days, in an easy way. It has similarities with Ruby, specially on syntax,
|
|
so Ruby developers are quite excited with the rapid growing of Elixir. A full-stack Ruby developer
|
|
can learn how to use Elixir and Phoenix in just a few weeks!
|
|
|
|
In Elixir we have a command called `mix`, which is a helper to create projects, testing, run
|
|
migrations and [much more](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix). We'll use it later on in this tutorial.
|
|
|
|
Check the [Elixir documentation](https://elixir-lang.org/getting-started/introduction) for more information.
|
|
|
|
## Requirements
|
|
|
|
To follow this tutorial, you'll need to have installed:
|
|
|
|
- Elixir [installation instructions](https://elixir-lang.org/install)
|
|
- Phoenix Framework [installation instructions](https://hexdocs.pm/phoenix/installation.html)
|
|
- PostgreSQL (if you need to use MySQL server, check [Phoenix instructions](https://hexdocs.pm/phoenix/ecto.html#using-mysql))
|
|
|
|
### Create a new Phoenix project
|
|
|
|
Open your terminal and go to the directory you wish to create your project.
|
|
You don't need to create an empty directory for the project's files, because the `mix` command will
|
|
do it for us.
|
|
|
|
When we call `mix` command, we'll pass two arguments:
|
|
|
|
- The task we want it to run: `phoenix.new`
|
|
- And the parameter `phoenix.new` requires, which is the name of the new project. In this case,
|
|
we're calling it `hello_gitlab_ci`, but you're free to set your own name:
|
|
|
|
```shell
|
|
mix phoenix.new hello_gitlab_ci
|
|
```
|
|
|
|
When asked, answer `Y` to fetch and install dependencies.
|
|
|
|
If everything went fine, you'll get an output like this:
|
|
|
|
![mix phoenix.new](img/mix-phoenix-new.png)
|
|
|
|
Now, our project is located inside the directory with the same name we pass to `mix` command, for
|
|
example, `~/GitLab/hello_gitlab_ci`.
|
|
If we take a look at the directory, we'll see the Phoenix files and the dependencies needed to run.
|
|
|
|
### Initialize the PostgreSQL database
|
|
|
|
By default, Phoenix requires a PostgreSQL database to store whatever we need to store in our app. In
|
|
this case, we'll only create an empty database.
|
|
|
|
First, we need to navigate to our recently created project's directory, and then execute again
|
|
`mix`. This time, `mix` will receive the parameter `ecto.create`, which is the task to create our
|
|
new database. [Ecto](https://hexdocs.pm/ecto/Ecto.html) is the database wrapper for Elixir.
|
|
|
|
When we do run `mix` the first time after creating our project, it will compile our files to
|
|
bytecode, which will be interpreted by Erlang VM. In the next times, it will only compile our
|
|
changes.
|
|
|
|
Run the commands below to create our empty database:
|
|
|
|
```shell
|
|
cd hello_gitlab_ci
|
|
mix ecto.create
|
|
```
|
|
|
|
We expect to see this output at the end of the command:
|
|
|
|
```plaintext
|
|
Generated hello_gitlab_ci app
|
|
The database for HelloGitlabCi.Repo has been created
|
|
```
|
|
|
|
NOTE: **Note:**
|
|
Phoenix assumes that our PostgreSQL database will have a `postgres` user account with the correct
|
|
permissions and a password of `postgres`. If it's not your case, check
|
|
[Ecto's instructions](https://hexdocs.pm/ecto/Ecto.html#module-repositories).
|
|
|
|
### Start Phoenix server
|
|
|
|
Now, it's time to see if everything we did until now went well. We'll call `mix` again, this time
|
|
with `phoenix.server` parameter, which will start Phoenix's HTTP Server.
|
|
|
|
```shell
|
|
mix phoenix.server
|
|
```
|
|
|
|
This will be the output to this command:
|
|
|
|
```plaintext
|
|
[info] Running HelloGitlabCi.Endpoint with Cowboy using http://localhost:4000
|
|
23 May 11:44:35 - info: compiling
|
|
23 May 11:44:37 - info: compiled 6 files into 2 files, copied 3 in 9.8 sec
|
|
```
|
|
|
|
Now, we have our app running locally. We can preview it directly on our browser. Let's open
|
|
[`localhost:4000`](http://localhost:4000) to see our Phoenix Framework welcome page. If the link do
|
|
not work, open [`127.0.0.1:4000`](http://127.0.0.1:4000) instead and later, configure your OS to
|
|
point `localhost` to `127.0.0.1`.
|
|
|
|
![mix phoenix.server](img/mix-phoenix-server.png)
|
|
|
|
Great, now we have a local Phoenix Server running our app.
|
|
|
|
Locally, our application is running in an [`iex`](https://elixir-lang.org/getting-started/introduction.html#interactive-mode) session, which stands for Interactive Elixir.
|
|
In this interactive mode, we can type any Elixir expression and get its result. To exit `iex`, we
|
|
need to press `Ctrl+C` twice. So, when we need to stop the Phoenix server, we have to hit `Ctrl+C`
|
|
twice.
|
|
|
|
## Introducing GitLab CI/CD
|
|
|
|
With GitLab, we can manage our development workflow, improve our productivity, track issues,
|
|
perform code review, and much more from a single platform. With GitLab CI/CD, we can be much more
|
|
productive, because every time we, or our co-workers push any code, GitLab CI/CD will build and
|
|
test the changes, telling us in real time if anything goes wrong.
|
|
|
|
Certainly, when our application starts to grow, we'll need more developers working on the same
|
|
project and this process of building and testing can easily become a mess without proper management.
|
|
That's also why GitLab CI/CD is so important to our application. Every time someone pushes its code to
|
|
GitLab, we'll quickly know if their changes broke something or not. We don't need to stop everything
|
|
we're doing to test manually and locally every change our team does.
|
|
|
|
Let's see this in practice.
|
|
|
|
## Adjusting Phoenix configuration
|
|
|
|
Now, we need to adjust our Phoenix configuration before configuring GitLab CI/CD.
|
|
There is a directory (`config`) in your Phoenix project that contains a configuration file for every
|
|
environment it can run. Since we will work with a single environment, we'll edit just the test
|
|
configuration file (`test.exs`).
|
|
|
|
But, why do we need to adjust our configuration? Well, GitLab CI/CD builds and tests our code in one
|
|
isolated virtual machine, called a [runner](../../runners/README.md), using Docker technology. In this runner,
|
|
GitLab CI/CD has access to everything our Phoenix application need to run, exactly as we have in our
|
|
`localhost`, but we have to tell GitLab CI/CD where to create and find this database using system
|
|
variables. This way, GitLab CI/CD will create our test database inside the runner, just like we do
|
|
when running our Phoenix in our `localhost`.
|
|
|
|
- Open `hello_gitlab_ci/config/test.exs` on your favorite code editor
|
|
- Go to **Configure your database** session and edit the block to include `System.get_env`:
|
|
|
|
```elixir
|
|
# Configure your database
|
|
config :hello_gitlab_ci, HelloGitlabCi.Repo,
|
|
adapter: Ecto.Adapters.Postgres,
|
|
username: System.get_env("POSTGRES_USER") || "postgres",
|
|
password: System.get_env("POSTGRES_PASSWORD") || "postgres",
|
|
database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test",
|
|
hostname: System.get_env("POSTGRES_HOST") || "localhost",
|
|
pool: Ecto.Adapters.SQL.Sandbox
|
|
```
|
|
|
|
We'll need these system variables later on.
|
|
|
|
- Create an empty file named `.gitkeep` into `hello_gitlab_ci/priv/repo/migrations`
|
|
|
|
As our project is still fresh, we don't have any data on our database, so, the `migrations`
|
|
directory will be empty.
|
|
Without `.gitkeep`, Git will not upload this empty directory and we'll got an error when running our
|
|
test on GitLab.
|
|
|
|
NOTE: **Note:**
|
|
If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir.
|
|
|
|
Now, let's run a local test and see if everything we did didn't break anything.
|
|
|
|
## Testing
|
|
|
|
Earlier, when we created our project, we ran `mix phoenix.new`.
|
|
This task created everything a Phoenix application needed, including some unit tests into
|
|
`hello_gitlab_ci/test` directory.
|
|
|
|
Let's run a new task with `mix` to run those tests for us. This time, the parameter expected is
|
|
`test`. We can add `--trace` parameter for debugging purposes.
|
|
|
|
In your terminal, navigate to the directory `hello_gitlab_ci` and run:
|
|
|
|
```shell
|
|
mix test
|
|
```
|
|
|
|
Our expected result is this:
|
|
|
|
```plaintext
|
|
....
|
|
|
|
Finished in 0.7 seconds
|
|
4 tests, 0 failures
|
|
|
|
Randomized with seed 610000
|
|
```
|
|
|
|
Our test was successful. It's time to push our files to GitLab.
|
|
|
|
## Configuring CI/CD Pipeline
|
|
|
|
The first step is to create a new file called `.gitlab-ci.yml` in `hello_gitlab_ci` directory of our
|
|
project.
|
|
|
|
- The easiest way to do this is to click on **Set up CI/CD** on project's main page:
|
|
|
|
![Set up CI](img/set_up_ci_v12_6.png)
|
|
|
|
- On the next screen, we can use a template with Elixir tests already included. Click on **Apply a template** and select **Elixir**:
|
|
|
|
![Select template](img/select_template_v12_6.png)
|
|
|
|
This template file tells GitLab CI/CD about what we wish to do every time a new commit is made.
|
|
However, we have to adapt it slightly to run a Phoenix app.
|
|
|
|
- The first line tells GitLab what Docker image will be used.
|
|
|
|
Remember when we learned about runners, the isolated virtual machine where GitLab CI/CD builds and tests
|
|
our application? This virtual machine must have all dependencies to run our application. This is
|
|
where a Docker image is needed. The correct image will provide the entire system for us.
|
|
|
|
As we are focusing on testing (not deploying), you can use the [elixir:latest](https://hub.docker.com/_/elixir) Docker image, which already has the
|
|
dependencies for running Phoenix tests installed, such as Elixir and Erlang:
|
|
|
|
```yaml
|
|
image: elixir:latest
|
|
```
|
|
|
|
- We'll only use `postgres`, so we can delete the `mysql` and `redis` lines from the `services` section:
|
|
|
|
```yaml
|
|
services:
|
|
- postgres:latest
|
|
```
|
|
|
|
- Now, we'll create a new section called `variables`, before the `before_script` section:
|
|
|
|
```yaml
|
|
variables:
|
|
POSTGRES_DB: hello_gitlab_ci_test
|
|
POSTGRES_HOST: postgres
|
|
POSTGRES_USER: postgres
|
|
POSTGRES_PASSWORD: "postgres"
|
|
MIX_ENV: "test"
|
|
```
|
|
|
|
Above, we set up the values for GitLab CI/CD to authenticate into PostgreSQL, like we did in
|
|
`config/test.exs` earlier. The `POSTGRES_USER` and `POSTGRES_PASSWORD` values
|
|
are used by the `postgres` service to create a user with those credentials.
|
|
|
|
- In the `before_script` section, we'll add some commands to prepare everything for the test:
|
|
|
|
```yaml
|
|
before_script:
|
|
- mix local.rebar --force
|
|
- mix local.hex --force
|
|
- mix deps.get --only test
|
|
- mix ecto.create
|
|
- mix ecto.migrate
|
|
```
|
|
|
|
This ensures that [rebar3](https://www.rebar3.org) and [hex](https://hex.pm) are both installed
|
|
before attempting to fetch the dependencies that are required to run the tests. Next, the `postgres` db
|
|
is created and migrated with `ecto`, to ensure it's up-to-date.
|
|
|
|
- Finally, we'll leave the `mix` section unchanged.
|
|
|
|
Let's take a look at the updated file after the changes:
|
|
|
|
```yaml
|
|
image: elixir:latest
|
|
|
|
services:
|
|
- postgres:latest
|
|
|
|
variables:
|
|
POSTGRES_DB: hello_gitlab_ci_test
|
|
POSTGRES_HOST: postgres
|
|
POSTGRES_USER: postgres
|
|
POSTGRES_PASSWORD: "postgres"
|
|
MIX_ENV: "test"
|
|
|
|
before_script:
|
|
- mix local.rebar --force
|
|
- mix local.hex --force
|
|
- mix deps.get --only test
|
|
- mix ecto.create
|
|
- mix ecto.migrate
|
|
|
|
mix:
|
|
script:
|
|
- mix test
|
|
```
|
|
|
|
For safety, we can check if we get any syntax errors before submitting this file to GitLab. Copy the
|
|
contents of `.gitlab-ci.yml` and paste it on [GitLab CI/CD Lint tool](https://gitlab.com/ci/lint). Please note that
|
|
this link will only work for logged in users.
|
|
|
|
## Watching the build
|
|
|
|
I don't know about you, but I love to watch that black screen being filled with compilation output.
|
|
With this, I can feel the happiness of something I made working correctly. On `localhost` it's easy
|
|
to watch our build, but on GitLab, is it possible? Yes!
|
|
|
|
Let's go to **Pipelines** and see GitLab doing the job. Just click on **Pipelines** to find the
|
|
actual running build job.
|
|
|
|
![Pipelines](img/pipelines.png)
|
|
|
|
Click on build's ID to watch the entire process. If everything went as expected, we can wait for the
|
|
**Build succeeded** at the end of the process! :)
|
|
|
|
```shell
|
|
$ mix test
|
|
....
|
|
|
|
Finished in 0.3 seconds
|
|
4 tests, 0 failures
|
|
|
|
Randomized with seed 206909
|
|
Build succeeded
|
|
```
|
|
|
|
If we take a look at the project's main page on the GitLab UI, we can see the status of the last
|
|
build made by GitLab CI/CD.
|
|
|
|
Time to show the world our green build badge! Navigate to your project's **Settings > CI/CD** and
|
|
expand **General pipelines settings**. Scroll down to **Pipeline status** and copy the Markdown code
|
|
for your badge. Paste it on the top of your `README.md` file, to let people outside of our project
|
|
see if our latest code is running without errors.
|
|
|
|
When we finish this edition, GitLab will start another build and show a **build running** badge. It
|
|
is expected, after all we just configured GitLab CI/CD to do this for every push! But you may think
|
|
"Why run build and tests for simple things like editing README.md?" and it is a good question.
|
|
For changes that don't affect your application, you can add the keyword [`[ci skip]`](../../yaml/README.md#skip-pipeline)
|
|
to commit message and the build related to that commit will be skipped.
|
|
|
|
In the end, we finally got our pretty green build succeeded badge! By outputting the result on the
|
|
README file, it shows to whoever lands on your project's page that your code is up-to-date and
|
|
working properly.
|
|
|
|
## Conclusion
|
|
|
|
When we have a growing application with many developers working on it, or when we have an open
|
|
source project being watched and contributed by the community, it is really important to have our
|
|
code permanently working. GitLab CI/CD is a time saving powerful tool to help us maintain our code
|
|
organized and working.
|
|
|
|
As we could see in this post, GitLab CI/CD is really easy to configure and use. We have [many
|
|
other reasons](https://about.gitlab.com/blog/2015/02/03/7-reasons-why-you-should-be-using-ci/) to keep
|
|
using GitLab CI/CD. The benefits to our teams will be huge!
|
|
|
|
## References
|
|
|
|
- [GitLab CI/CD introductory guide](https://about.gitlab.com/blog/2015/12/14/getting-started-with-gitlab-and-gitlab-ci/)
|
|
- [GitLab CI/CD full Documentation](../../README.md)
|
|
- [GitLab Runner documentation](../../runners/README.md)
|
|
- [Using Docker images documentation](../../docker/using_docker_images.md)
|