Commit graph

543 commits

Author SHA1 Message Date
NigeParis
98631be918 WIP BroadCast working on console... needs more work 2025-11-21 14:17:37 +01:00
NigeParis
d89c21dd2c WIP on separation socket-client from chat.ts 2025-11-21 14:17:37 +01:00
NigeParis
9889600708 separated socket from app for clear develoopement 2025-11-21 14:17:37 +01:00
Maieul BOYER
0a504a75ce socket io with fastify 2025-11-21 14:17:37 +01:00
NigeParis
34e9f8e3ca socket default acess refuse server 2025-11-21 14:17:37 +01:00
NigeParis
cf6f3145b6 socket.io first attempt 2025-11-21 14:17:37 +01:00
Maieul BOYER
5a905a1239 feat(oauth2/db): reworked oauth2 database footprint
- Removed `auth` table and merged its information inside the `user`
  table
- Changed around some field names in the database
- Changed Create*User functions to not be using overload but different
  functions
2025-11-18 15:30:55 +01:00
Maieul BOYER
34249bf68d
chore(deps): removed vulnerable dependency (rimraf) 2025-11-18 15:18:07 +01:00
Maieul BOYER
b1d4f68453 feat(frontend): added returnTo to login and signin page 2025-11-16 17:06:40 +01:00
Raphaël
033d399fcb
Merge pull request #38 from Maix0/maix/fixes/monitoring 2025-11-15 11:49:43 +01:00
Maieul BOYER
a4bf71cc6a
feat(monitoring): wrote plugin for /monitoring endpoint and updated service to use them 2025-11-14 22:43:31 +01:00
Raphaël
e44a3af76d
Logs module
## 🦌 Centralized Logging Stack Integration

### ELK Stack Online

- Added **`elasticsearch`**, **`logstash`**, and **`kibana`** services to `docker-compose.yml`:
  - **Elasticsearch** for log storage and indexing with persistent volumes.
  - **Logstash** as the GELF entrypoint, handling log ingestion and transformation.
  - **Kibana** as the web UI for log exploration, dashboards, and saved searches.
- Each ELK service is wired with:
  - **Persistent storage** to survive restarts.
  - **Environment variables** for credentials and tuning.
  - **Bootstrap scripts** to perform initial setup (policies, templates, dashboards, etc.).

### Global GELF Logging

- All existing services now use the **GELF logging driver** in `docker-compose.yml`:
  - Containers send their logs to **Logstash** instead of stdout-only.
  - Provides **structured**, centralized logs ready for querying in Elasticsearch/Kibana.
- Result: no more log hunting across containers — everything lands in one searchable place.

---

## 🔁 Log Lifecycle & Visualization Automation

### Elasticsearch & Kibana Bootstrap

- Introduced **bootstrap scripts and config files** to automate:
  - **Index Lifecycle Management (ILM)** policies for log retention and rollover.
  - **Index templates** for log indices (naming, mappings, and settings).
  - **Kibana imports** (index patterns / data views, dashboards, visualizations).
- This turns ELK setup from a manual ritual into a **single-command provisioning step**.

### Logstash Pipeline Upgrade

- Added a **Logstash pipeline configuration** to:
  - Ingest **GELF** logs from Docker.
  - **Normalize/rename fields** for consistent querying across services.
  - Index logs into **Elasticsearch** with **daily rotation per container** pattern.
- Outcome: logs are structured, tagged by container, and auto-rotated to keep storage sane.

---

## 🛠 Makefile & Docker.mk Enhancements

### Logs Setup Targets

- Added a new **`logs`** target in `Makefile` (with `.PHONY` declaration) to manage logging setup from the top level.
- Added a **`logs-setup`** target in `Docker.mk` to:
  - Initialize **ILM policies** in Elasticsearch.
  - Apply **index templates** for logs.
  - Create **Kibana index patterns** so logs are immediately visible in the UI.
- These targets plug into the existing tooling, making logging setup part of the **standard dev/ops workflow**.

---

## 🔐 Environment Configuration

### Secure Elasticsearch Access

- Updated `env.example` to include:
  - **`ELASTIC_PASSWORD`**: central password for Elasticsearch authentication.
- Encourages **secure-by-default** deployments and aligns local/dev with production-style security.

---

## 📈 Monitoring Configuration Updates

### Grafana Alerting & Prometheus Cleanup

- Added a **basic alerting policy for Grafana**:
  - Provides a default routing tree for alerts.
  - Acts as a foundation for future, more granular alert rules.
- Cleaned up **Prometheus scrape configuration**:
  - Removed obsolete backend scrape targets.
  - Keeps monitoring config focused on **live** and relevant services.
2025-11-14 22:01:24 +01:00
Raphael
f1805ff9be
Merge branch 'master' into raph/logs 2025-11-14 21:53:46 +01:00
Maieul BOYER
f1569b7120
chore(deps): updated pnpm-lock.yaml 2025-11-14 19:10:20 +01:00
Maieul BOYER
470007c9f8
feat(chat): removed unused depedencies 2025-11-14 19:08:27 +01:00
Maieul BOYER
7bdf667ece
feat(logs): removed unused mkdir in elasticsearch 2025-11-14 18:54:39 +01:00
Maieul BOYER
6a75faa40f
feat(logs/kibana): made it so dashboard is default page 2025-11-14 18:32:28 +01:00
Maieul BOYER
46fb7d81aa
feat(logs/kibana): import dashboard 2025-11-14 18:27:40 +01:00
Maieul BOYER
8dd0a74046
feat(logs): make it so rule logs-setup isn't necessary 2025-11-14 17:58:45 +01:00
Maieul BOYER
6d630fee92 feat(frontend/auth): Added way to automatically fetch providers
This allows the frontend to list all available providers without
hardcoding them in.
2025-11-14 16:57:14 +01:00
Raphaël
3ef811b334
Merge branch 'master' into raph/logs 2025-11-14 16:25:06 +01:00
Raphael
b6789188ae
core(env): adding the elastic password to the env 2025-11-14 16:21:49 +01:00
Raphael
bf0b1a7772
feat(logs/ilm): adding the logs management 2025-11-14 16:18:01 +01:00
Raphael
9adc92f549
feat(logs/ilm): adding the memory management 2025-11-14 16:17:51 +01:00
Raphael
2f4d55e119
feat(logs/logstash): adding the logstash configuration 2025-11-14 16:17:23 +01:00
Raphael
7356041e0d
build(compose): adding the logging rules 2025-11-14 16:16:59 +01:00
Raphael
c4aae6b9dd
feat(logs): Adding the log containers 2025-11-14 16:16:42 +01:00
Raphael
9f7d88da41
build(make): adding the logs-setup rules 2025-11-14 16:15:21 +01:00
Raphael
3302710ae0
build(docker/monitoring): adding the logs part to the makefile 2025-11-14 16:14:57 +01:00
NigeParis
9ce9fa44e4 feat(chat): added base for chat service
added front html

nigel in the mud

route function with openapi - gen

clean up the code a little

after pull request
2025-11-14 15:50:03 +01:00
Raphael
21b4db5a56
refactor(montoring/prometheus): removing the useless endpoint 2025-11-13 23:09:36 +01:00
Raphael
9e98ff2d13
monitoring(monitoring/grafana): setting up the rules for the notification 2025-11-13 23:07:38 +01:00
Raphael
20cac0e42b
feat(monitoring/grafana): setting up the notification policies 2025-11-13 23:07:08 +01:00
Raphaël
73a4946d17
Monitoring tools (#32)
* feat(docker/monitoring): adding the first monitoring tools from the docker file

- The first tools is Grafana basicly the tool to supervise all the data

* feat(monitoring/blackbox): adding initial configuration

* feat(monitoring/grafana): adding the configuration to alerting on the discord channels

* feat(monitoring/grafana): adding the grafana dashboard (docker monitoring)

* feat(monitoring/grafana): adding the grafana dashboard (global monitoring)

* feat(monitoring/grafana): adding the global configuration for dashboards

* feat(monitoring/grafana): adding the prometheus configuration

* feat(monitoring/prometheus): adding the configuration of prometheus as the main grafana sources

* core(docker-compose): adding the monitoring part for the docker files

* feat(monitoring/grafana): removing the monitoring global

* feat(monitoring/prometheus): removing the blackbox

- The self certificate is ruinning everything

* core(docker-compose): removing the blackbox container

* core(env/example): adding a env example

* feat(monitoring/blackbox): adding initial configuration

* test(nginx/location): adding a test to test blackbox

* feat(monitoring/prometheus): adding blackbox to the prometheus configuration

* feat(monitoring/grafana): adding the start of the global dashboard

* feat(monitoring/blackbox): adding tls_configuration skip

- The ssl certificate have to be self-signed

* feat(monitoring/grafana): global is now checked and work w/ others services

* feat(monitoring/prometheus): checking other services run

* feat(nginx/conf): now http port will mirror the https

- Usefull for intern docker communication

* feat(auth/app): adding the /monitoring routes

* feat(icons/app): adding the /monitoring routes

* feat(user/app): adding the /monitoring routes

* refactor(auth/src): linting the app.ts

* refactor(icons/src): linting the app.ts

* refactor(user/src): linting the app.ts
2025-11-12 11:50:32 +01:00
Maieul BOYER
fac6df3882 fixes(frontend): fixed small types/incorrect code 2025-11-11 00:20:26 +01:00
Maieul BOYER
16cac7e111 chore(git): add package-lock.json to gitignore 2025-11-11 00:20:26 +01:00
Maieul BOYER
aba4c4498c feat(frontend/auth): fix cookie not working due to path being /app
Cookies being set to path=/app meant that the API didn't have those
cookies.
Also fixed the Schema injection for auth'ed routes
2025-11-11 00:20:26 +01:00
Maieul BOYER
e8b0b7e310 feat(frontend/openapi): Generated Openapi client for frontend
Generated The API client using the openapi.json file generated from the
services
2025-11-11 00:20:26 +01:00
Maieul BOYER
08c910c193 feat(frontend): added frontend
- Router: client side route handling with client side rendering
- Toast: rought Toast handling for better UX and messaging
- Auth: single point of truth for the Logged in user

This commit doesnt not include the openapi generated code
2025-11-11 00:20:26 +01:00
Maieul BOYER
0db41a440d feat(frontend): scaffolding to build frontend
- Added `make fnginx` to have better dev experience
- Added app.conf to nginx configuration to handle frontend
2025-11-11 00:20:26 +01:00
Maieul BOYER
5dd6067c95 feat(openapi): Add modified typescript-fetch template
This template is modified from the original one, to handle multiple
status-code handling of response.

Don't ask me how they work, I don't quite understand them in depth...
2025-11-10 18:34:22 +01:00
Maieul BOYER
b7c2a3dff9 feat(openapi): Started working on Openapi generation
- Updated to Typebox 1.0.0 to better support Openapi type generation
- Changed dockerfile to fetch depedencies only once
- Fixed Routes to properly handle openapi
- Fixed Routes to respond with multiples status code (no more only 200)
- Fixed Schemas so the auth-gated endpoint properly reflect that
- Added Makefile rule to generate openapi client (none working due to
  missing files)
2025-11-10 18:34:22 +01:00
Raphael
1bd2b4594b fix(direnv/envrc): adding the use flake to the repo 2025-11-03 15:58:39 +01:00
Raphael
8ebd6b6bff feat(flake/tmux-setup): adding the tmux-setup w/ lazydocker 2025-11-03 15:58:39 +01:00
Maieul BOYER
fc2cfdb7cf
chore(Makefile): reworked npm@ rules to use pnpm 2025-10-29 19:38:31 +01:00
Maieul BOYER
a66fad3360
chore(lockfile/icons): updated lockfile and fixed icons service shutdown
- Updated lockfile and packages version
- Updated icons service to stop on SIGTERM
2025-10-29 19:24:59 +01:00
Maieul BOYER
332086d5e2 feat(auth): split login_name and display_name for better oauth2/guest user handling 2025-10-28 13:47:20 +01:00
Maieul BOYER
e0689143c4 feat(oauth2): fixed small issues 2025-10-28 13:47:20 +01:00
Maieul BOYER
a67d29af3c feat(oauth2): commented a bit of stuff 2025-10-28 13:47:20 +01:00
Maieul BOYER
66537e2ccb feat(oauth2/user): Implementation of the whole oauth2 pipeline
Auth: user can now login with any oauth2 provider the config list
      account will be created on demands if not already created.
      any oauth2 account is unique (can't link an account to any other
      auth method)
2025-10-28 13:47:20 +01:00