You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
docuseal/docs/qa/assessments/1.2.core-models-implementat...

378 lines
13 KiB

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

# Risk Profile: Story 1.2 - Core Models Implementation
**Assessment Date:** 2026-01-15
**QA Agent:** Quinn (Test Architect & Quality Advisor)
**Overall Risk Score:** 42/100 (Lower is better)
---
## Executive Summary
This risk assessment identifies 12 potential risks across 6 categories for Story 1.2 (Core Models Implementation). The story involves creating 4 ActiveRecord models with a 7-state machine, implementing feature flag protection, and integrating with existing DocuSeal tables.
**Risk Distribution:**
- **Critical (Score 9):** 0 risks
- **High (Score 6):** 5 risks
- **Medium (Score 4):** 2 risks
- **Low (Score 2-3):** 4 risks
- **Minimal (Score 1):** 1 risk
**Total Risk Score:** 42/100
---
## Risk Matrix
| Risk ID | Description | Category | Probability | Impact | Score | Priority |
|---------|-------------|----------|-------------|--------|-------|----------|
| **TECH-001** | State machine complexity - 7 states with complex transitions | TECH | Medium (2) | High (3) | **6** | High |
| **TECH-002** | AASM gem integration issues or configuration errors | TECH | Low (1) | Medium (2) | **2** | Low |
| **SEC-001** | Feature flag bypass - FloDoc routes not properly protected | SEC | Medium (2) | High (3) | **6** | High |
| **SEC-002** | Email validation gaps on sponsor_email/student_email | SEC | Low (1) | Medium (2) | **2** | Low |
| **PERF-001** | N+1 queries on model associations (institution→cohorts→enrollments) | PERF | High (3) | Medium (2) | **6** | High |
| **PERF-002** | Missing database indexes on frequently queried columns | PERF | Medium (2) | Medium (2) | **4** | Medium |
| **DATA-001** | Foreign key constraint violations with existing tables | DATA | Medium (2) | High (3) | **6** | High |
| **DATA-002** | JSONB field validation failures (required_student_uploads, cohort_metadata) | DATA | Low (1) | Medium (2) | **2** | Low |
| **DATA-003** | Unique constraint violations (cohort_enrollments.submission_id) | DATA | Low (1) | High (3) | **3** | Low |
| **BUS-001** | State machine logic doesn't match business workflow | BUS | Medium (2) | High (3) | **6** | High |
| **OPS-001** | Feature flag seed data missing or incorrect | OPS | Low (1) | Low (1) | **1** | Minimal |
| **OPS-002** | Test coverage below 80% target | OPS | Medium (2) | Medium (2) | **4** | Medium |
---
## Critical Risks (Score 6)
### 1. TECH-001: State Machine Complexity
**Score: 6 (High)**
**Probability**: Medium - Complex state transitions with 7 states
**Impact**: High - Incorrect workflow could block business operations
**Description**: The Cohort model implements a 7-state machine (draft → tp_signed → students_completed → sponsor_completed → finalized → active → completed) with complex transition rules. Missing guard clauses or incorrect transitions could cause data integrity issues.
**Mitigation Strategy**:
- Implement comprehensive state transition tests for all valid/invalid transitions
- Add guard clauses for state transitions (e.g., cannot skip steps)
- Document state machine diagram in code comments
- Test edge cases: concurrent state changes, rollback scenarios
**Testing Focus**:
- Unit tests for all state transition events (10+ scenarios)
- Integration tests for complete workflow (draft → completed)
- Edge case: Invalid transitions should raise errors
- Concurrency tests for simultaneous state changes
---
### 2. SEC-001: Feature Flag Bypass
**Score: 6 (High)**
**Probability**: Medium - Missing before_action in controllers
**Impact**: High - FloDoc functionality exposed prematurely
**Description**: FloDoc routes must be protected by feature flag checks. Missing protection could expose functionality before it's ready for production.
**Mitigation Strategy**:
- Implement FeatureFlagCheck concern with require_feature helper
- Add controller specs that verify feature flag protection
- Test both enabled and disabled states
- Create integration test for full request flow
**Testing Focus**:
- Controller specs with feature flag enabled/disabled
- Request specs verifying 404/403 when flag disabled
- Test feature flag toggle functionality
- Verify all FloDoc routes are protected
---
### 3. PERF-001: N+1 Query Issues
**Score: 6 (High)**
**Probability**: High - Common issue with nested associations
**Impact**: Medium - Performance degradation with 1000+ records
**Description**: Models have nested associations (institution→cohorts→enrollments). Without proper eager loading, queries could cause N+1 performance issues.
**Mitigation Strategy**:
- Use `includes()` or `eager_load()` for all association queries
- Add performance tests with 1000+ test records
- Use Bullet gem or similar to detect N+1 queries
- Verify with EXPLAIN queries
**Testing Focus**:
- Performance tests with large datasets (1000+ records)
- Query optimization verification
- Association loading tests
- EXPLAIN query analysis for slow queries
---
### 4. DATA-001: Foreign Key Constraint Violations
**Score: 6 (High)**
**Probability**: Medium - Referencing non-existent template/submission IDs
**Impact**: High - Data integrity issues, failed saves
**Description**: Models reference existing DocuSeal tables (templates, submissions). Foreign key constraints could prevent saves if referenced records don't exist.
**Mitigation Strategy**:
- Validate foreign key existence before save
- Create test helpers for dependent records
- Add database-level foreign key constraints
- Test rollback scenarios
**Testing Focus**:
- Integration tests with real foreign key references
- Test data integrity with missing references
- Verify FK constraints prevent orphaned records
- Test cascading delete/soft delete behavior
---
### 5. BUS-001: State Machine Logic Mismatch
**Score: 6 (High)**
**Probability**: Medium - Business requirements vs implementation
**Impact**: High - Workflow doesn't match business needs
**Description**: State machine implementation must match PRD business requirements. Mismatch could cause workflow failures.
**Mitigation Strategy**:
- Validate state machine against PRD requirements
- Get business stakeholder review of state transitions
- Document business rules for each state
- Add acceptance criteria tests for state transitions
**Testing Focus**:
- Business requirement validation tests
- State transition approval tests
- Workflow completion tests
- PRD requirement traceability tests
---
## Medium Risks (Score 4)
### 6. PERF-002: Missing Database Indexes
**Score: 4 (Medium)**
**Probability**: Medium - Indexes not added on queried columns
**Impact**: Medium - Query performance degradation
**Mitigation Strategy**:
- Add indexes on all foreign keys
- Add indexes on frequently queried columns (status, email)
- Verify index usage with EXPLAIN queries
- Test query performance with large datasets
**Testing Focus**:
- Database migration specs for index creation
- EXPLAIN query analysis
- Performance tests with 1000+ records
---
### 7. OPS-002: Test Coverage Below 80%
**Score: 4 (Medium)**
**Probability**: Medium - Insufficient test coverage
**Impact**: Medium - Quality issues, bugs in production
**Mitigation Strategy**:
- Calculate test coverage after implementation
- Add missing test scenarios
- Use coverage tools (SimpleCov, RCov)
- Ensure >80% coverage requirement is met
**Testing Focus**:
- Unit test coverage for all models
- Integration test coverage for workflows
- Feature flag protection tests
- State machine transition tests
---
## Low Risks (Score 1-3)
### 8. DATA-003: Unique Constraint Violations
**Score: 3 (Low)**
**Probability**: Low - Duplicate submission_id
**Impact**: High - Data integrity issues
**Mitigation Strategy**:
- Add unique constraint on cohort_enrollments.submission_id
- Test duplicate submission handling
- Verify constraint prevents duplicates
**Testing Focus**:
- Unit tests for unique constraint
- Integration tests for duplicate prevention
- Error handling for constraint violations
---
### 9. SEC-002: Email Validation Gaps
**Score: 2 (Low)**
**Probability**: Low - Missing format validation
**Impact**: Medium - Invalid email data
**Mitigation Strategy**:
- Add email format validation to all email fields
- Test valid/invalid email formats
- Verify validation errors are raised
**Testing Focus**:
- Unit tests for email validation
- Integration tests for email format checking
---
### 10. DATA-002: JSONB Field Validation Failures
**Score: 2 (Low)**
**Probability**: Low - Invalid JSON data
**Impact**: Medium - Data corruption
**Mitigation Strategy**:
- Add JSON schema validation for complex fields
- Test valid/invalid JSON data
- Verify validation errors are raised
**Testing Focus**:
- Unit tests for JSONB field validation
- Integration tests for data integrity
---
### 11. TECH-002: AASM Gem Integration Issues
**Score: 2 (Low)**
**Probability**: Low - Gem configuration errors
**Impact**: Medium - State machine not working
**Mitigation Strategy**:
- Verify AASM gem installation and configuration
- Test state machine initialization
- Verify event callbacks work correctly
**Testing Focus**:
- Unit tests for AASM configuration
- Integration tests for state machine functionality
---
## Minimal Risks (Score 1)
### 12. OPS-001: Feature Flag Seed Data Missing
**Score: 1 (Minimal)**
**Probability**: Low - Seed data not created
**Impact**: Low - Feature flag not available
**Mitigation Strategy**:
- Create seed data for feature flags
- Test seed data creation
- Verify feature flags exist in database
**Testing Focus**:
- Seed data tests
- Feature flag availability tests
---
## Risk-Based Testing Strategy
### Priority 1: Critical Risk Tests (Score 6+)
1. **State Machine Tests** - All 7 states, all transitions (TECH-001, BUS-001)
2. **Feature Flag Protection Tests** - Controller/request level (SEC-001)
3. **Foreign Key Constraint Tests** - Integration with existing tables (DATA-001)
4. **N+1 Query Detection Tests** - Performance with 1000+ records (PERF-001)
5. **Business Workflow Validation Tests** - State transitions match PRD (BUS-001)
### Priority 2: High Risk Tests (Score 4)
1. **Database Index Tests** - Verify indexes on queried columns (PERF-002)
2. **Test Coverage Verification** - >80% coverage requirement (OPS-002)
### Priority 3: Medium/Low Risk Tests (Score 1-3)
1. **Email Validation Tests** - Format validation on all email fields (SEC-002)
2. **JSONB Field Tests** - Validation of complex fields (DATA-002)
3. **Unique Constraint Tests** - submission_id uniqueness (DATA-003)
4. **Feature Flag Seed Tests** - Default flags present (OPS-001)
---
## Risk Acceptance Criteria
### Must Fix Before Production
- All critical risks (score 6) must be mitigated
- State machine must pass all transition tests
- Feature flag protection must be verified
- Foreign key constraints must be tested
- Test coverage must exceed 80%
### Can Deploy with Mitigation
- Medium risks (score 4) with compensating controls
- Low risks (score 2-3) with monitoring in place
### Accepted Risks
- Minimal risks (score 1) can be accepted with documentation
- Performance optimization can be deferred if within NFR limits
---
## Monitoring Requirements
Post-deployment monitoring for:
- **Performance metrics** - Query times with 1000+ records
- **Error rates** - State machine transition failures
- **Feature flag usage** - Toggle frequency and impact
- **Data integrity** - Foreign key constraint violations
---
## Risk Review Triggers
Review and update risk profile when:
- State machine requirements change
- New associations added to models
- Feature flag system modified
- Performance issues reported in production
- Business workflow changes
---
## Gate YAML Block Output
```yaml
risk_summary:
totals:
critical: 0 # score 9
high: 5 # score 6
medium: 2 # score 4
low: 4 # score 2-3
minimal: 1 # score 1
highest:
id: TECH-001
score: 6
title: 'State machine complexity - 7 states with complex transitions'
recommendations:
must_fix:
- 'Implement comprehensive state transition tests for all valid/invalid transitions'
- 'Add FeatureFlagCheck concern with require_feature helper in controllers'
- 'Use includes() or eager_load() for all association queries to prevent N+1'
- 'Validate foreign key existence before save with test helpers'
- 'Verify state machine logic matches PRD business requirements'
monitor:
- 'Monitor query performance with 1000+ records post-deployment'
- 'Track feature flag toggle frequency and errors'
- 'Alert on state machine transition failures'
- 'Monitor foreign key constraint violations'
```
---
## Key Principles Applied
**Risk-Based Testing** - Focused on high-impact areas
**Probability × Impact** - Systematic scoring (6 high, 2 medium, 5 low)
**Actionable Mitigation** - Specific testing strategies for each risk
**Gate-Ready Output** - YAML format for quality gate integration
**Business Alignment** - State machine validation against PRD
---
**Risk Score: 42/100** (Lower is better - 100 = no risk)
**Recommendation:** Address all 5 high-risk items before implementation. The state machine complexity and feature flag protection are the most critical risks that could block production deployment.