Website : linking with frontend & backend

realaravinth commented

2022-03-15 10:20:27 +05:30

Owner

No description provided.

realaravinth added this to the MVP milestone 2022-03-15 10:20:27 +05:30

realaravinth added the

billing

priority-1

labels 2022-03-15 10:20:27 +05:30

realaravinth self-assigned this 2022-03-15 10:20:27 +05:30

realaravinth added this to the July MVP project 2022-03-15 10:20:27 +05:30

dachary commented

2022-03-20 06:34:31 +05:30

Owner

Here is the basic logic for the backend to communicate with the frontend. We discussed using webhooks as a way to trigger actions in the backend but we did not discuss what these webhooks should do. The section below, "Updating the Gitea instances", explains what they need to do, ultimately.

If the frontend triggers a webhook, it needs to send information via JSON, the script watching the webhook then receives the JSON, parses it, writes YAML files that are meaningful to Ansible and then calls Ansible to interpret these YAML files and create/update/destory the Gitea instances.

Alternatively, the frontend could write the YAML files directly and call Ansible which would save steps and there would be no need to define an intermediary JSON format that has no purpose.

If that makes sense to you, the next step will be for me to document which YAML files must contain what so the frontend can create them to reflect the state of the Gitea instances controled by a given user.

Updating the Gitea instances

The creation, update and deletion of a Gitea instance is done by running ansible with a set of variables. The frontend needs to create files that contain these variables. For instance changing gitea.yml from:

---
gitea_version: 1.16.3

to

---
gitea_version: 1.16.4

Will upgrade the gitea instance to version 1.16.4 the next time Ansible runs. The frontend should not run Ansible directly though but use the enough command line instead: it wraps Ansible with sensible defaults.

Getting information about the Gitea instances

All Gitea instances are monitored individually by Icinga. In addition, the icinga probes archive values in an influxdb database which linked with grafana. To get information about an instance, the frontend can:

Query enough (to get the IP for instance)
Query Icinga (to know which services are up or not)
Query influxdb (to get disk usage, network usage, etc.)
Query grafana (to get graphs on disk usage, network usage, etc.)

It can then populate a dashboard displayed to the user when it is logged in, with all informations regarding the past and present resources it rented.

Here is the basic logic for the backend to communicate with the frontend. We discussed using webhooks as a way to trigger actions in the backend but we did not discuss what these webhooks should do. The section below, "Updating the Gitea instances", explains what they need to do, ultimately. If the frontend triggers a webhook, it needs to send information via JSON, the script watching the webhook then receives the JSON, parses it, writes YAML files that are meaningful to Ansible and then calls Ansible to interpret these YAML files and create/update/destory the Gitea instances. Alternatively, the frontend could write the YAML files directly and call Ansible which would save steps and there would be no need to define an intermediary JSON format that has no purpose. If that makes sense to you, the next step will be for me to document which YAML files must contain what so the frontend can create them to reflect the state of the Gitea instances controled by a given user. ### Updating the Gitea instances The creation, update and deletion of a Gitea instance is done by running ansible with a set of variables. The frontend needs to create files that contain these variables. For instance changing `gitea.yml` from: ``` --- gitea_version: 1.16.3 ``` to ``` --- gitea_version: 1.16.4 ``` Will upgrade the gitea instance to version 1.16.4 the next time Ansible runs. The frontend should not run Ansible directly though but use the enough command line instead: it wraps Ansible with sensible defaults. ### Getting information about the Gitea instances All Gitea instances are monitored individually by Icinga. In addition, the icinga probes archive values in an influxdb database which linked with grafana. To get information about an instance, the frontend can: * Query enough (to get the IP for instance) * Query Icinga (to know which services are up or not) * Query influxdb (to get disk usage, network usage, etc.) * Query grafana (to get graphs on disk usage, network usage, etc.) It can then populate a dashboard displayed to the user when it is logged in, with all informations regarding the past and present resources it rented.

dachary added spent time 2022-03-20 06:34:47 +05:30

30min

dachary self-assigned this 2022-03-20 07:08:51 +05:30

realaravinth added spent time 2022-03-20 14:24:48 +05:30

2h

realaravinth commented

2022-03-20 14:47:07 +05:30

Author

Owner

By "frontend" I'm assuming you are talking about the payments software and the "backend" is the lifecycle software.

I understand that the lifecycle can be handled entirely by the payments server, but I would prefer to keep the payments server as lean and as generic as possible. Also, I prefer to JSON for all communications involving the payments server. Adding YAML would mean maintenance overhead.

Getting information about the Gitea instances

All Gitea instances are monitored individually by Icinga. In
addition, the icinga probes archive values in an influxdb database which
linked with grafana. To get information about an instance, the frontend
can

I was under the impression that payments would run at payments.example.com and the lifecycle backend at dash.example.com. This way, the user will interact with the payments software only for payments. If the functionality can't be implemented within the lifecycle backend, then I propose that we create a separate dashboard program.

By "frontend" I'm assuming you are talking about the payments software and the "backend" is the lifecycle software. I understand that the lifecycle can be handled entirely by the payments server, but I would prefer to keep the payments server as lean and as generic as possible. Also, I prefer to JSON for all communications involving the payments server. Adding YAML would mean maintenance overhead. > ## Getting information about the Gitea instances > All Gitea instances are monitored individually by Icinga. In addition, the icinga probes archive values in an influxdb database which linked with grafana. To get information about an instance, the frontend can I was under the impression that payments would run at [payments.example.com](https://payments.example.com) and the lifecycle backend at [dash.example.com](https://dash.example.com). This way, the user will interact with the payments software only for payments. If the functionality can't be implemented within the lifecycle backend, then I propose that we create a separate dashboard program.

dachary changed title from ~~Billing : linking with website & devops backend~~ to Website : linking with website & devops backend

2022-03-20 15:10:01 +05:30

dachary added

frontend

and removed

billing

labels 2022-03-20 15:13:35 +05:30

dachary added the

backend

label 2022-03-20 15:19:44 +05:30

dachary changed title from ~~Website : linking with website & devops backend~~ to Website : linking with frontend & backend

2022-03-20 15:20:06 +05:30

dachary commented

2022-03-20 16:04:24 +05:30

Owner

Regarding how the frontend sends instructions to the backend, here is how we will move forward:

(i) I'll document the files the frontend needs to produce and their semantic,
(ii) you make it so the frontend creates those files to reflect what the user wants
(iii) I'll make it so those files are acted upon.

In other words I'll make it so you don't need to understand the underlying Ansible logic but that it is really equivalent to sending a payload to a webhook that says: "create this VM with this amount of ram and run gitea + ci on it". Only it's not a webhook, it's a plain file

Regarding how the frontend sends instructions to the backend, here is how we will move forward: (i) I'll document the files the frontend needs to produce and their semantic, (ii) you make it so the frontend creates those files to reflect what the user wants (iii) I'll make it so those files are acted upon. In other words I'll make it so you don't need to understand the underlying Ansible logic but that it is really equivalent to sending a payload to a webhook that says: "create this VM with this amount of ram and run gitea + ci on it". Only it's not a webhook, it's a plain file

👍 1

dachary commented

2022-03-20 16:14:48 +05:30

Owner

By "frontend" I'm assuming you are talking about the payments software and the "backend" is the lifecycle software.

For the record we clarified the website / devops / billing relationship: website + billing are part of the "frontend" and the "website" label was renamed "frontend". The "devops" label was renamed "backend" and is about everything needed to run the infrastructure, based on enough.community.

@realaravinth thanks for your patience, this was rather confusing and I feel we now have clarity on how the pieces fit together.

> By "frontend" I'm assuming you are talking about the payments software and the "backend" is the lifecycle software. For the record we clarified the website / devops / billing relationship: website + billing are part of the "frontend" and the "website" label was renamed "frontend". The "devops" label was renamed "backend" and is about everything needed to run the infrastructure, based on enough.community. @realaravinth thanks for your patience, this was rather confusing and I feel we now have clarity on how the pieces fit together.

❤️ 1

dachary added spent time 2022-03-20 16:15:01 +05:30

1h 30min

dachary commented

2022-03-20 18:43:32 +05:30

Owner

https://icinga.hostea.org is up and has icinga + grafana + influx DB

🚀 1

dachary added spent time 2022-03-20 18:44:00 +05:30

45min

dachary commented

2022-03-22 04:17:30 +05:30

Owner

To create a new instance named foobar-host, the following file must be added to ~/.enough/hostea.org/inventory/host_vars/foobar-host/gitea.yml file, with the values of the variable set according to the user preferences.

---
#
#######################################
#
# Hostname of the Gitea instance
#
#
gitea_host: "gitea.{{ domain }}"
#
#######################################
#
# Mailer from
#
#
gitea_mailer_from: "noreply@enough.community"
#
#######################################
#
# SSH port of the Gitea instance
#
#
gitea_ssh_port: "22"
#
#######################################
#
# Gitea version
#
#
gitea_version: "1.16.4"
#
#######################################
#
# Admin user name
#
gitea_user: root
#
#######################################
#
# Admin user password
#
gitea_password: etquofEtseudett
#
#######################################
#
# Admin user email
#
gitea_email: contact@enough.community

The size of the VM hosting the gitea instance is set in the ~/.enough/hostea.org/inventory/host_vars/foobar-host/provision.yml file:

---
libvirt_ram: 8192
libvirt_disk: 20G
libvirt_cpus: 6

To delete an instance the ~/.enough/hostea.org/inventory/host_vars/foobar-host directory must be removed and the foobar-host name added to the list of deleted hosts in the ~/.enough/hostea.org/inventory/deleted.yml file:

---
deleted-hosts:
  hosts:
    foobar-host:

This is necessary so that monitoring knows to delete all resources associated with foobar-host on the next update.

To create a new instance named `foobar-host`, the following file must be added to `~/.enough/hostea.org/inventory/host_vars/foobar-host/gitea.yml` file, with the values of the variable set according to the user preferences. ``` --- # ####################################### # # Hostname of the Gitea instance # # gitea_host: "gitea.{{ domain }}" # ####################################### # # Mailer from # # gitea_mailer_from: "noreply@enough.community" # ####################################### # # SSH port of the Gitea instance # # gitea_ssh_port: "22" # ####################################### # # Gitea version # # gitea_version: "1.16.4" # ####################################### # # Admin user name # gitea_user: root # ####################################### # # Admin user password # gitea_password: etquofEtseudett # ####################################### # # Admin user email # gitea_email: contact@enough.community ``` The size of the VM hosting the gitea instance is set in the `~/.enough/hostea.org/inventory/host_vars/foobar-host/provision.yml` file: ``` --- libvirt_ram: 8192 libvirt_disk: 20G libvirt_cpus: 6 ``` To delete an instance the `~/.enough/hostea.org/inventory/host_vars/foobar-host` directory must be removed and the `foobar-host` name added to the list of deleted hosts in the `~/.enough/hostea.org/inventory/deleted.yml` file: ``` --- deleted-hosts: hosts: foobar-host: ``` This is necessary so that monitoring knows to delete all resources associated with `foobar-host` on the next update.

🚀 1

dachary added spent time 2022-03-22 04:17:45 +05:30

15min

Website : linking with frontend & backend #12

Updating the Gitea instances

Getting information about the Gitea instances

Getting information about the Gitea instances