diff --git a/load-test/README.md b/load-test/README.md index c9335fa..1dae5a9 100644 --- a/load-test/README.md +++ b/load-test/README.md @@ -14,26 +14,27 @@ source ./env/bin/activate # 1. Install dependency pip install -r requirements.txt -# 2. Generate test data dari content/ -python content_parser.py +# 2. Generate test data dari content/ (konten ada di subfolder dasar/, arduino/, circuit/) +python content_parser.py --content-dir ../../content --tokens-file ../../tokens_siswa.csv --num-tokens 50 # 3. Jalankan Locust (opsional: set VELXIO_HOST jika Velxio bukan di localhost:8001) export VELXIO_HOST=http://localhost:8001 locust -f locustfile.py ``` -Buka **http://localhost:8089**, masukkan URL backend Elemes (misalnya `http://localhost:5000`), lalu mulai test. +Buka **http://localhost:8089**, masukkan URL backend Elemes (misalnya `http://localhost:3000`), lalu mulai test. ## File | File | Fungsi | |------|--------| -| `content_parser.py` | Parse `content/*.md` → `test_data.json` + inject token test ke CSV | -| `locustfile.py` | Locust script (7 task weighted) yang baca `test_data.json` | +| `content_parser.py` | Parse `content/**/*.md` (rekursif subfolder) → `test_data.json` + inject token `LOCUST_TEST_*` ke CSV | +| `locustfile.py` | Locust script (task weighted) yang baca `test_data.json` | | `test_data.json` | Auto-generated, **jangan di-commit** | | `requirements.txt` | Dependency (`locust`) | +| `e2e_session_check.py` | Smoke-test alur interaktif (PTY session) via Flask proxy — jalan di container | -## Test Scenarios (8 Tasks) +## Test Scenarios (9 Tasks) | # | Task | Weight | Target | Deskripsi | |---|------|--------|--------|----------| @@ -41,17 +42,63 @@ Buka **http://localhost:8089**, masukkan URL backend Elemes (misalnya `http://lo | 2 | View Detail | 5 | Elemes | GET `/lesson/{slug}.json`, validasi field per tipe | | 3 | Compile C | 4 | Elemes | POST `/compile`, validasi output vs expected | | 4 | Compile Python | 3 | Elemes | POST `/compile`, validasi output vs expected | -| 5 | Verify Arduino | 2 | Elemes | GET lesson Arduino, validasi JSON structure | -| 6 | Complete Flow | 2 | Both | fetch → compile → track-progress | -| 7 | Progress Report | 1 | Elemes | Login guru → GET `/progress-report.json` | -| 8 | **Compile Arduino** | **3** | **Velxio** | POST `/api/compile`, validasi hex_content | +| 5 | **Interactive Session Python** | **4** | **Elemes** | POST `/compile/sessions` → poll prompt → POST `/input` → verifikasi output → DELETE | +| 6 | Verify Arduino | 2 | Elemes | GET lesson Arduino, validasi JSON structure | +| 7 | Complete Flow | 2 | Both | fetch → compile → track-progress | +| 8 | Progress Report | 1 | Elemes | Login guru → GET `/progress-report.json` | +| 9 | **Compile Arduino** | **3** | **Velxio** | POST `/api/compile`, validasi hex_content | + +> **Task 5 (interactive session)** menguji alur baru playground: Run → prompt muncul → ketik jawaban → Enter → output. Skrip memakai token (login) agar lolos rate-limit anonymous dan benar-benar mengukur kapasitas compiler worker (max 50 sesi, 2 compile bersamaan, queue timeout 20 s). + +## Load Test 50 Pengguna (2 Device) + +Host hanya 4-core/3.5 GiB — 50 kompilasi berat bersamaan tidak realistis, tapi 50 sesi yang **menunggu input** aman (proses idle tidak makan CPU). Batas worker: `INTERACTIVE_MAX_SESSIONS=50`, `INTERACTIVE_MAX_COMPILES=2`, `INTERACTIVE_QUEUE_TIMEOUT_SECONDS=20`. + +### Device A — Locust Master (jalankan UI di sini) + +```bash +cd elemes/load-test +source ./env/bin/activate +locust -f locustfile.py --master --web-port 8089 +``` + +### Device B (atau beberapa device) — Locust Worker + +```bash +cd elemes/load-test +source ./env/bin/activate +# Ganti dengan IP device A (di jaringan/Tailscale yang sama) +locust -f locustfile.py --worker --master-host +``` + +### Mulai Test + +1. Buka `http://:8089` di browser device A. +2. **Host**: URL app yang diuji, misal `https://sinau-c-dev.manakin-gentoo.ts.net` (Tailscale Funnel) atau `http://localhost:3000`. +3. **Number of users**: 50 · **Spawn rate**: 5–10/s (ramp-up ~5–10 detik agar hampir bersamaan) · **Duration**: 5–10 menit. +4. Amati di tab Charts: RPS per endpoint, response time `/compile/sessions` & `/input`, dan failure rate. +5. Skenario `/compile/sessions [Python interactive]` wajib ≥95% sukses; failure `429` pada create menandakan anon rate limit (bukan kapasitas) — gunakan token. + +> Catatan: task compile biasa (`/compile`) tetap kena anon rate limit `1 per 2 menit` bila token tidak ikut — login di `on_start` menyimpan cookie, jadi user ber-token exempt. ## Re-generate Setelah Tambah Lesson Baru Setiap kali ada lesson baru di `content/`, cukup jalankan ulang: ```bash -python content_parser.py +python content_parser.py --content-dir ../../content --tokens-file ../../tokens_siswa.csv ``` `test_data.json` akan di-update otomatis dan Locust langsung test lesson baru. + +## E2E Smoke Test (container) + +Verifikasi cepat alur interaktif tanpa browser — jalan di dalam container backend: + +```bash +podman cp elemes/load-test/e2e_session_check.py lms-dev_elemes_1:/tmp/ +podman exec -e E2E_TOKEN= lms-dev_elemes_1 python3 /tmp/e2e_session_check.py +# Contoh token: LOCUST_TEST_xxxx (ambil dari tokens_siswa.csv) +``` + +Tanpa `E2E_TOKEN` skrip tetap jalan tapi hanya 2 create session (anon rate limit 1/2 menit). diff --git a/load-test/content_parser.py b/load-test/content_parser.py index a4e697d..9484b8b 100644 --- a/load-test/content_parser.py +++ b/load-test/content_parser.py @@ -62,6 +62,15 @@ def detect_lesson_type(content: str) -> str: return 'c' +def find_lesson_file(content_dir: str, slug: str) -> str | None: + """Cari file .md dengan slug (basename) di content_dir secara rekursif.""" + target = f'{slug}.md' + for root, _dirs, files in os.walk(content_dir): + if target in files: + return os.path.join(root, target) + return None + + def parse_lesson(filepath: str) -> dict: """Parse a single lesson markdown file and extract test data.""" with open(filepath, 'r', encoding='utf-8') as f: @@ -123,6 +132,22 @@ def parse_lesson(filepath: str) -> dict: return data +def scan_all_lessons(content_dir: str) -> list[str]: + """Scan recursively for lesson slugs (basename without .md). + + Konten tersimpan di subfolder (dasar/, arduino/, circuit/), sedangkan + API lesson memakai slug tanpa folder (find_lesson_file mencari di semua + folder). Sub-home.md bukan lesson, di-skip. + """ + slugs = [] + for root, _dirs, files in os.walk(content_dir): + for f in sorted(files): + if not f.endswith('.md') or f in ('home.md', 'sub-home.md'): + continue + slugs.append(f[:-3]) + return sorted(set(slugs)) + + def get_ordered_slugs(content_dir: str) -> list[str]: """Get lesson slugs in order from home.md's Available_Lessons section.""" home_path = os.path.join(content_dir, 'home.md') @@ -239,21 +264,23 @@ def main(): # 1. Get ordered lesson slugs ordered_slugs = get_ordered_slugs(content_dir) + all_slugs = scan_all_lessons(content_dir) if not ordered_slugs: - # Fallback: scan directory - ordered_slugs = [ - f.replace('.md', '') - for f in sorted(os.listdir(content_dir)) - if f.endswith('.md') and f not in ('home.md', 'sub-home.md') - ] + # Fallback: recursive scan (konten di subfolder) + ordered_slugs = all_slugs + else: + # Gabungkan: slug dari home.md (urutan) + slug hasil scan yang belum ada + for s in all_slugs: + if s not in ordered_slugs: + ordered_slugs.append(s) print(f" 📚 Found {len(ordered_slugs)} lessons:") # 2. Parse each lesson lessons = [] for slug in ordered_slugs: - filepath = os.path.join(content_dir, f'{slug}.md') - if not os.path.exists(filepath): + filepath = find_lesson_file(content_dir, slug) + if not filepath: print(f" ⚠ {slug}.md not found, skipping") continue diff --git a/load-test/e2e_session_check.py b/load-test/e2e_session_check.py new file mode 100644 index 0000000..f930d3e --- /dev/null +++ b/load-test/e2e_session_check.py @@ -0,0 +1,139 @@ +#!/usr/bin/env python3 +"""E2E check alur interaktif playground via proxy Flask (di dalam container).""" +import json +import sys +import time +import urllib.request +import urllib.error + +BASE = "http://127.0.0.1:5000" +TOKEN = __import__("os").environ.get("E2E_TOKEN", "").strip() + + +def _with_token(payload): + if TOKEN: + payload = dict(payload) + payload["token"] = TOKEN + return payload + + +def call(method, path, payload=None, params=None): + url = BASE + path + if params: + url += "?" + "&".join(f"{k}={v}" for k, v in params.items()) + data = json.dumps(payload).encode() if payload is not None else None + req = urllib.request.Request(url, data=data, method=method) + if data: + req.add_header("Content-Type", "application/json") + try: + with urllib.request.urlopen(req, timeout=15) as r: + return r.status, json.loads(r.read().decode()) + except urllib.error.HTTPError as e: + body = e.read().decode() + try: + return e.code, json.loads(body) + except Exception: + return e.code, {"raw": body[:200]} + + +def wait_for(sid, predicate, timeout=15, poll=0.3, cursor=0): + """Poll output; return (status_dict, final_cursor).""" + last = None + t0 = time.time() + while time.time() - t0 < timeout: + st, body = call("GET", f"/compile/sessions/{sid}", params={"cursor": cursor}) + if st != 200: + last = ("ERR", st, body) + time.sleep(poll) + continue + cursor = body.get("cursor", cursor) + last = body + if predicate(body): + return body, cursor + time.sleep(poll) + return last, cursor + + +results = [] + + +def check(name, cond, detail=""): + results.append((name, bool(cond), detail)) + print(f"{'PASS' if cond else 'FAIL'} {name}" + (f" {detail}" if detail and not cond else "")) + + +# ── 1. Python interaktif: Run → prompt → input → output ────────────── +st, body = call("POST", "/compile/sessions", _with_token({ + "language": "python", + "files": [{"name": "main.py", "content": 'nama = input("Siapa nama kamu? ")\nprint(f"Halo, {nama}!")\n'}], + "active_file": "main.py", +})) +check("py: create session", st in (200, 202) and body.get("session_id"), f"st={st} body={body}") +sid = body.get("session_id", "") + +if sid: + b, cur = wait_for(sid, lambda x: "Siapa nama kamu?" in x.get("output", "")) + check("py: prompt muncul (tanpa stdin, tetap running)", b.get("status") == "running" and "Siapa nama kamu?" in b.get("output", ""), f"{b.get('status')} out={b.get('output')!r}") + + st, b2 = call("POST", f"/compile/sessions/{sid}/input", {"text": "anggoro"}) + check("py: input kirim OK", st == 200, f"st={st}") + + b3, _ = wait_for(sid, lambda x: x.get("status") in ("exited", "error"), cursor=cur) + out = b3.get("output", "") + check("py: output Halo, anggoro!", "Halo, anggoro!" in out and b3.get("exit_code") == 0, f"status={b3.get('status')} exit={b3.get('exit_code')} out={out!r}") + check("py: tidak ada EOFError", "EOFError" not in out, out[-200:]) + + st, _ = call("DELETE", f"/compile/sessions/{sid}") + check("py: delete idempotent", st in (200, 404), f"st={st}") + +# ── 2. C interaktif: scanf prompt → input → output ─────────────────── +C_PROMPT = '#include \nint main() {\n char nama[64];\n printf("Nama: ");\n fflush(stdout);\n scanf("%63s", nama);\n printf("Halo, %s!\\n", nama);\n return 0;\n}\n' +st, body = call("POST", "/compile/sessions", _with_token({ + "language": "c", + "files": [{"name": "main.c", "content": C_PROMPT}], + "active_file": "main.c", +})) +check("c: create session", st in (200, 202) and body.get("session_id"), f"st={st} body={body}") +sid = body.get("session_id", "") +if sid: + b, cur = wait_for(sid, lambda x: "Nama:" in x.get("output", ""), timeout=20) + check("c: prompt Nama: muncul", b.get("status") == "running" and "Nama:" in b.get("output", ""), f"{b.get('status')} out={b.get('output')!r}") + st, _ = call("POST", f"/compile/sessions/{sid}/input", {"text": "Budi"}) + b3, _ = wait_for(sid, lambda x: x.get("status") in ("exited", "error"), cursor=cur, timeout=20) + check("c: output Halo, Budi!", "Halo, Budi!" in b3.get("output", "") and b3.get("exit_code") == 0, f"status={b3.get('status')} out={b3.get('output')!r}") + call("DELETE", f"/compile/sessions/{sid}") + +# ── 3. C multi-file dengan foo.h (include header) ──────────────────── +FILES = [ + {"name": "main.c", "content": '#include \n#include "foo.h"\nint main() {\n printf("Hasil: %d\\n", tambah(7, 8));\n return 0;\n}\n'}, + {"name": "foo.h", "content": "#ifndef FOO_H\n#define FOO_H\nstatic inline int tambah(int a, int b) { return a + b; }\n#endif\n"}, +] +st, body = call("POST", "/compile/sessions", _with_token({"language": "c", "files": FILES, "active_file": "main.c"})) +check("c-multi: create session", st in (200, 202) and body.get("session_id"), f"st={st} body={body}") +sid = body.get("session_id", "") +if sid: + b, _ = wait_for(sid, lambda x: x.get("status") in ("exited", "error"), timeout=25) + out = b.get("output", "") + check("c-multi: foo.h ikut dikompilasi (Hasil: 15)", "Hasil: 15" in out and b.get("exit_code") == 0, f"status={b.get('status')} out={out!r}") + +# ── 4. Python tanpa stdin: tetap aktif, bukan EOFError ─────────────── +st, body = call("POST", "/compile/sessions", _with_token({ + "language": "python", + "files": [{"name": "main.py", "content": 'x = input("Angka: ")\nprint(x)\n'}], +})) +sid = body.get("session_id", "") +if sid: + b, _ = wait_for(sid, lambda x: "Angka:" in x.get("output", ""), timeout=15) + check("py-nostdin: prompt tetap aktif (bukan EOF)", b.get("status") == "running", f"status={b.get('status')}") + call("DELETE", f"/compile/sessions/{sid}") + +# ── 5. Error handling: sesi invalid → 404 ──────────────────────────── +st, body = call("GET", "/compile/sessions/tidak-ada") +check("404 session unknown", st == 404, f"st={st}") + +fails = [r for r in results if not r[1]] +print("\n===== HASIL =====") +print(f"{len(results) - len(fails)}/{len(results)} PASS") +if fails: + print("FAIL:", [(r[0], r[2]) for r in fails]) + sys.exit(1) diff --git a/load-test/locustfile.py b/load-test/locustfile.py index 33fb0b7..a575723 100644 --- a/load-test/locustfile.py +++ b/load-test/locustfile.py @@ -248,6 +248,113 @@ class ElemesStudent(HttpUser): f"not in output '{output[:50]}'" ) + # ── Task 4b: Interactive Session Python (weight=4) ──────────────── + # Alur baru playground: Run → prompt → ketik jawaban → Enter → output. + + @task(4) + def interactive_session_python(self): + """Buat sesi interaktif Python, kirim input, verifikasi output. + + Memakai token (login) agar lolos rate-limit anonymous dan mengukur + kapasitas compiler worker (max 50 sesi, 2 compile bersamaan). + """ + code = ( + 'nama = input("Siapa nama kamu? ")\n' + 'print(f"Halo, {nama}! Selamat belajar pemrograman.")\n' + ) + payload = { + 'language': 'python', + 'files': [{'name': 'main.py', 'content': code}], + 'active_file': 'main.py', + 'token': self.token, + } + + with self.client.post( + f'{API}/compile/sessions', + json=payload, + name='/compile/sessions [Python interactive]', + catch_response=True, + timeout=30, + ) as resp: + data = safe_json(resp) + if resp.status_code not in (200, 202) or not data.get('session_id'): + resp.failure( + f"Session create failed: HTTP {resp.status_code} " + f"{data.get('error', '')}" + ) + return + session_id = data['session_id'] + + # Poll sampai prompt muncul (menandakan program menunggu input) + cursor = 0 + prompt_seen = False + for _ in range(40): # 40 x 0.5s = 20s maks + time.sleep(0.5) + with self.client.get( + f"{API}/compile/sessions/{session_id}", + params={'cursor': cursor}, + name='/compile/sessions/{id} [poll]', + catch_response=True, + timeout=10, + ) as resp: + poll = safe_json(resp) + if resp.status_code != 200: + resp.failure(f"Poll failed: HTTP {resp.status_code}") + break + cursor = poll.get('cursor', cursor) + if 'Siapa nama kamu?' in poll.get('output', ''): + prompt_seen = True + break + + if not prompt_seen: + self.client.delete( + f"{API}/compile/sessions/{session_id}", + name='/compile/sessions/{id} [cleanup]', + ) + return + + # Kirim input (tanpa newline — worker menambahkannya) + with self.client.post( + f"{API}/compile/sessions/{session_id}/input", + json={'text': 'Budi'}, + name='/compile/sessions/{id}/input', + catch_response=True, + timeout=10, + ) as resp: + if resp.status_code != 200: + resp.failure(f"Input failed: HTTP {resp.status_code}") + + # Poll sampai selesai; verifikasi output + for _ in range(40): + time.sleep(0.5) + with self.client.get( + f"{API}/compile/sessions/{session_id}", + params={'cursor': cursor}, + name='/compile/sessions/{id} [poll]', + catch_response=True, + timeout=10, + ) as resp: + poll = safe_json(resp) + if resp.status_code != 200: + resp.failure(f"Poll failed: HTTP {resp.status_code}") + break + cursor = poll.get('cursor', cursor) + status = poll.get('status') + if status in ('exited', 'error', 'stopped'): + output = poll.get('output', '') + if 'Halo, Budi!' not in output: + resp.failure( + f"Output mismatch (status={status}): " + f"{output[-120:]!r}" + ) + break + + # Cleanup session (best-effort) + self.client.delete( + f"{API}/compile/sessions/{session_id}", + name='/compile/sessions/{id} [cleanup]', + ) + # ── Task 5: Verify Arduino Lesson Structure (weight=2) ───────────── @task(2) diff --git a/velxio b/velxio index 6e17819..2b17470 160000 --- a/velxio +++ b/velxio @@ -1 +1 @@ -Subproject commit 6e1781926e5988ac5eb6ea51a30a4e8f945beec7 +Subproject commit 2b1747041b06a67db8d758908fd54d516af86927