Hostea/july-mvp
Archived
3
2
Fork 1
Code Issues 28 Pull requests Projects 1 Releases Wiki Activity

add TODO pad

Browse source
This commit is contained in:
Aravinth Manivannan 2022-03-14 23:38:30 +05:30
parent c8cffff93c
commit 7055d4fb7f
Signed by: realaravinth
GPG key ID: AD9F0F08E855ED88
1 changed files with 274 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

274
README.md Normal file
View file

@ -0,0 +1,274 @@
# TODO(July MVP)
- [x] check the task list (@aravinth)
- [ ] set a number of hours next to each task (@loic @aravinth)
- [ ] sort the tasks in the timeline so that it fits (@loic @aravinth)
---
# Scope
## Personae
### Admin and billing
Creates the instance, pays the bills, has admin rights allowing them to get backups, configure the DNS, terminate the instance, is the contact point for legal matters in case of abuse.
### User
Has an account on the gitea instance for administration purposes or is a non-privileged user.
### Sysadmin
Creates the hosting service, connects it to a resource provider (for instance OVH to get VM), connects it to a payment provider (for instance a stripe account linked to a bank account), answers support requests from Admin and billing, upgrades the hosting service.
## Use cases
### Hosting service Creation
### Hosting service connection to a resource provider
### Hosting service connection to a payment provider
### Hosting service support request
### Instance Creation
### Instance DNS
### Instance Grow
### Instance Backup
### Instance Termination
### Instance Billing
### Static pages
Tasks:
- [Umbrella task tracking all related subtasks](https://tasks.batsense.net/tasks/269)
- [Custom domain routing](https://tasks.batsense.net/tasks/268)
### Shared Instance:
In the shared instance business model the customer will be paying for an account, rather than a separate instance.
#### Motivation:
Managed hosting is expensive and not everyone will be able to pay for it. A complete forge experience includes:
- CI/CD
- Static page hosting
- Docker registry
- Pastebin
Hosting exclusive instances of all the above mentioned services can be very expensive. With shared hosting, customers will get the same forge experience for a modest subscription fee.
#### Challenges:
- [@loïc stated that shared instance will pose moderation challenges as the provider will be responsible for the content hosted on it](https://matrix.to/#/!QdcnJXkpkBVpecQGtk:matrix.batsense.net/$0xJpEALLE46l-4UiybUaIBbThMvVdAGtfVPi9hxewow?via=matrix.batsense.net&via=matrix.org&via=t2bot.io)
- Payments integration and account side effects(activation, termination, etc.)
#### References:
- [Matrix discussion](https://matrix.to/#/!QdcnJXkpkBVpecQGtk:matrix.batsense.net/$dR8nLlqcfZVAwwZkW5MafP2CvqYIzipemiJb3OOQWMY?via=matrix.batsense.net&via=matrix.org&via=t2bot.io)
### CI/CD
### Docker registry
### GitPad: pastebin implementation
### Abuse
# Task list
## User Research
@loic: [User Research](https://jdittrich.github.io/userNeedResearchBook/) is arguably the most useful step in the software development lifecycle and also almost always overlooked. In a nutshell it aims at increasing, by an order of magnitude, the odds that the software being developped matches the expectations of the users. Even when the users are the developers of the software itself.
### Question: why and how does someone create and use a self hosted Gitea hosting service? (loic: 2h, aravinth: 2h) (owner: loic)
@loic: this focuses on the experience of the sysadmin personae rather than the "Admin and billing" because most of the work done in the task list is for their benefit. If there was enough time, it would be useful to do user research based on a question focused on "Admin and billing". But given the limited resources we have, I think reasonable to narrow the focus in this way.
### Use Cases and Personae (loic: 3h, aravinth: 3h) (owner: loic)
@loic: the Use Cases above roughly list what the service is going to provide, from the point of view of each personae. It will be necessary to explain them with scenarios in the style of: "As a sysadmin... I will do this and I expect that".
### Interviews (loic: 16h, aravinth: 5h) (owner: loic)
@loic: the first step is to write interview scripts. The goal is that the interviewee tells us about its past experience. The script is guided by the question and the use cases: it helps drive the conversation so that the interviewee tells about his personal experience in the hope that it indirectly provides answers to the "why and how" question. Three interviewees must be recruited, appointments made, interview done, sessions recorded and transcribed.
### Analysis (loic: 5h, aravinth: 5h) (owner: loic, aravinth)
@loic: since this analysis cannot be done in person with actual post-its, it can be done online, during half a day. It essentially consists of grouping statements from all three interviews. They are narrative in nature and by isolating statements we want to break free of this narrative. We do that separately and then we discuss on how to merge those groups and why. And then we assign them priorities.
### Report (loic: 2h, aravinth: 5h) (owner: loic, aravinth)
@loic: this is a summary of the analysis session above that does not present the opinion of the writer. It summarizes the findings of the session and explains it based on what is found in the interviews. If something cannot be grounded in phrases found in the interviews, it is not part of the report. The outcome is a list of themes, sorted by priority. It tells the developer about what matters most to the user targetted by the question that is central to the user research, in this case the sysadmin. It is also a good way to explain to newcomes why the project went in this direction and not another, why it was felt more important to work on this aspect rather than another.
## Website
### Logo (loic: 2h, aravinth: 2h) (owner: aravinth)
@aravinth: I hired a friend to create logos for mCaptcha(unpublished) and ForgeFlux, we can do the same to save time. He charges 1000INR(~13 Euros)/engagement.
### Name (loic: 4h, aravinth: 2h) (owner: loic, aravinth)
### Design (loic: 5h, aravinth: 10h) (owner: loic, aravinth)
@loic: my estimate is based on the idea that the web design is subcontracted via 99designs.com, upwork etc.
@aravinth: my estimate is based on in-house design
### Customer workflow (loic: 50h, aravinth: ?) (owner: loic)
@loic: this is everything required for a paying subscriber to subscribe and control the service that is rented, with the exception of the payment system part which is taken care of by the Payment System part of the Billing section. This includes web based workflow for account registration, password reminder, termination of the service, upgrading to get a more powerful machine, setting the DNS, etc. It is also where users can control the backups of their instances.
@aravinth's understanding of customer workflow: enables the [sysadmin](https://pad.batsense.net/uPJSDVjRQhiy77du1Xgfvw?both#Sysadmin) to spin up and maintain a forge instance and manage their account.
## Sysadmin web control pannel
@loic: I propose there is none for the MVP, that's it is all CLI based.
## Billing
This covers the development of the software and documentation required to allow a sysadmin to successfully deploy the billing part of the gitea hosted service. It includes whatever is necessary to control the actions of a paying client depending on its balance.
### Payment system (loic: 120h, aravinth: 100h) (owner: aravinth)
@aravinth: the payment system will have the following features:
- generating custom invoices based on parameters supplied by sysadmins
- managing coupons and discount offers
- managing credit
- grace period before account is marked expired
- hooks to execute duing payment cycle events. Some of the events include:
- pre-account expiry
- pospt-account expiry
- coupon cashed
- payment success/failure
- grace period begins for account
- account running out of grace period
As part of the MVP, support will be implemented for Stripe integration
### Linking with website and devops backend (loic: 40h, aravinth: 20h) (owner: loic, aravinth)
@aravinth: this includes utilizing hooks that are provided by the payments system to effect changes in various services that are offered
### Documentation (loic: 24h) (owner: loic, aravinth)
@aravinth: documentation should include instructions to setup payments system in both Docker and bare metal<sup>\*</sup> setups and optionally include instructions to implement integration with a new payments provider(like Stripe)
[\*] I strongly believe we should include bare metal deployment instructions to avoid Discourse-like situations, where the only possible deployment is using Docker.
@loic: documentation includes detailed instructions that must be verified and tested multiple times to actually work as expected to setup the payment system. That is, for instance, create a stripe account, with screenshots, configure it in this way, with screenshots, get the token in this way with screenshots, copy the token in this file, etc. This is the only part of the payment system that cannot be made fully transparent to the sysadmin.
@aravinth: Agreed, my point supplements @loic's but what I meant was that, we should provide at least two deployment options(Docker and bare metal), instead of one([like Discourse](https://github.com/discourse/discourse/blob/main/docs/INSTALL.md#why-do-you-only-officially-support-docker))
## Architecture & Planning (loic: 8h, aravinth: 20h) (owner: loic, aravinth)
## Devops
This covers the development of the software and documentation required to allow a sysadmin to successfully deploy the gitea hosted service.
### Gitea instance lifecycle backend (loic: 40h, aravinth: 20h) (owner: loic, aravinth)
@loic: this covers the creation of a gitea instance on behalf of a paying user. It is driven by the CRM: a new instance is created when a new user exists and has permissions to run a gitea instance. If the instance has a DNS setup for this user, the instance is configured to answer this DNS. If a user no longer has permission to run a gitea instance, it is suspended. It also takes care of backups and recovery.
### Hosting Gitea instance lifecycle (loic: 40h, aravinth: 40h) (owner: loic)
@loic: this is the infrastructure as code the sysadmin will use when spawning a new hosting gitea service from scratch. They will follow instructions from the documentation and run commands from the CLI. When they are done they will get a new hosting gitea instance running at example.com and paying clients will be able to go there, subscribe to the service and get their gitea instance after paying with a credit card. This is also the infrastructure as code the sysadmin will use to upgrade their hosting gitea instance when a new release is published, with security patches etc or new features.
### Ticket system for support (loic: 2h, aravinth: 20h) (owner: loic, aravinth)
@loic: this is work needed to run a ticket system for customers to file issues. It should be linked with the CRM so that only paying customers can file issues. But in the context of a MVP it is enough that a ticket system exists and there does not need to be restrictions: it can be open to anyone, reason why I think the time to make this happen is low
### End to end testing (loic: 80h, aravinth: 50h) (owner: loic)
@loic: these are automated (emphasis on automated) end to end tests that (i) spawn a hosting gitea instance from scratch (ii) run scripts for each target use case (see use case). It may be quicker to do manual testing the first time around. But it will very quickly require documenting and running complex, lengthy manual testing. Or (and that's what really happens) there won't be proper testing and the whole system will very quickly be prone to regression and accumulate a technical debt it won't be able to pay in the very near future. Fortunately it can build on tooling created in the context of the enough.community project which has such end to end testing.
### CI/CD (loic: 16h, aravinth: 10h) (owner: loic, aravinth)
@aravinth: my estimate includes my prior experience with Woodpecker CI and our need to write/improve already existing glue code for Gitea<-->Woodpecker integration. It does not include "ops" side of things(openstack? libvirt?).
@loic: my estimate is based on my previous experience with Drone
### Static site hosting(loic: 40h, aravinth: 40h)(owner: aravinth, loic)
@aravinth: estimate break down:
- custom domain routing: 5h
- config parsing: 5h
- Gitea integration: 10h
- ACME implementaiton: 20h
### Shared hosting(loic: 124h, aravinth: 60h)(owner: aravinth)
#### system-wide per account resource usage monitoring: (loic: 24h, aravinth: 15h)
@aravinth: I'm not sure how this works, it is possible to monitor storage but what about compute power for CI/CD?
@loic: there is no tooling for handling quotas in gitea, it will need specific development. It can be an external script monitoring just the content of the gitea repositories owned by a given user. It gets more complicated for organizations though.
#### data takeout/download (loic: 0h, aravinth: 30h)
@aravinth: I think support present in Gitea would be sufficient but I wonder how integrations are managed. Woodpecker CI uses webhooks to talk to Gitea, I'm not sure if Gitea exports webhooks as well. If it does, we'll probably have to search and replace old-woodpecker.example.com with new-woodpecker.example.com
@loic: in the context of a MVP I think the existing migration process in Gitea is good enough.
#### payments integration (loic: 100h, aravinth: 15h)
@loic: the integration of permissions based on payment control in an automated way within the existing gitea codebase is significant work. I don't think it can be done exclusively via external scripts because it involves restricting what a user can do based on external information which is not something Gitea knows how to do at the moment. For instance if a user tries to login to the shared gitea instance, it should be notified that its subscription is about to expire and be offered a link to renew it. This could possibly be done by delegating authentication to an OAuth provider that enforces this policy. It would not require modifying gitea but it would require writing and running such a provider. In other words I'm not sure how to approach this but I sense the number of hours is closer to 100h than to 10h.
@aravinth: In Gitea, a user can have the following states:
- activated
- restricted
- admin
- sign in disabled
A crude payments integration can simply restrict(no idea what a restricted user's capabilities are) or can be denied sign in.
### Docker registry hosting ~~(loic: 24h, aravinth: ?) (owner:?)~~
@loic: I've never done that so I have no clue how to approach it. I would first have to research the simplest way to do that which is likely to take a full day with trial and errors. And then write an Ansible playbook to spawn the service. And then write and run tests to ensure it works as expected. I'm rather optimistic about this because it is likely something that comes up often and there maybe are simple solutions to self host a docker registry.
**REMOVED:** [@KN4CK3R(Janis) is working on implementing Docker registry in Gitea](https://github.com/go-gitea/gitea/issues/18345#issuecomment-1018247015)
### Documentation (loic: 24h) (owner: loic)
@loic: I would improve on the existing enough.community documentation which already targets the intended audience. There will be a "quick start part" with ideally a oneliner to setup the service for the first time. It may look like three full days to write a quick start is a long time. But from experience it is a feedback loop with development: the first time around the "quick start" is three pages long and requires 10 steps. Figuring out how to reduce the number of steps requires development so that it is sensible for the user and practical for the implementor.
### Gitpad (@aravinth: 5h)(owner: @aravinth)
@aravinth: GitPad is MVP-ready. Code snippets uploads and comments are ready as of 14/03/2022 and only documentation is pending.
## Operations
This covers provisioning and maintaining the resources required to develop and distribute the self hosted gitea hosting service.
### Website (loic: 2h, aravinth: 10h) (owner: loic, aravinth)
@loic: this includes all the resources to run the infrastructure (bind, postfix, icinga, nginx) and the gitea hosting service web frontend that customers can use to rent gitea instances. It is small because it is fully automated via infrastructure as code: the bulk of the work is on the implementation of the infrastructure as code which makes the operation part seem very light.
### Cloud provider procurement (loic: 1h, aravinth: 10h) (owner: loic)
@loic: opening an account at OVH.
### ~~Hardware procurement (loic: 2h, aravinth: 10h) (owner: loic, aravinth)~~
### Release management (loic: 8h, aravinth: 24h) (owner: loic, aravinth)
@loic: I assume the software stack + documentation will be python based because that's the language we're both familiar with. And there will be some docker image wrapping this because the devops part of things has lots of dependencies, in the same way enough.community does. The release management is about publishing the software stack on pypi, the documentation on rtd and the docker images on the docker hub.
# Timeline
## Static pages: 15/03 - 31/03(@aravinth)
- [ ] Custom domain routing(15/03 - 23/03)
- [ ] Configuration parsing from repository (23/03 - 24/03)
- [ ] ACME implementation for TLS certificates (25/03)
- [ ] Website preview on pull requests (26/03 - 31/03)