|
|
# Risk Profile: Story 1.2 - Core Models Implementation
|
|
|
|
|
|
**Assessment Date:** 2026-01-15
|
|
|
**QA Agent:** Quinn (Test Architect & Quality Advisor)
|
|
|
**Overall Risk Score:** 42/100 (Lower is better)
|
|
|
|
|
|
---
|
|
|
|
|
|
## Executive Summary
|
|
|
|
|
|
This risk assessment identifies 12 potential risks across 6 categories for Story 1.2 (Core Models Implementation). The story involves creating 4 ActiveRecord models with a 7-state machine, implementing feature flag protection, and integrating with existing DocuSeal tables.
|
|
|
|
|
|
**Risk Distribution:**
|
|
|
- **Critical (Score 9):** 0 risks
|
|
|
- **High (Score 6):** 5 risks
|
|
|
- **Medium (Score 4):** 2 risks
|
|
|
- **Low (Score 2-3):** 4 risks
|
|
|
- **Minimal (Score 1):** 1 risk
|
|
|
|
|
|
**Total Risk Score:** 42/100
|
|
|
|
|
|
---
|
|
|
|
|
|
## Risk Matrix
|
|
|
|
|
|
| Risk ID | Description | Category | Probability | Impact | Score | Priority |
|
|
|
|---------|-------------|----------|-------------|--------|-------|----------|
|
|
|
| **TECH-001** | State machine complexity - 7 states with complex transitions | TECH | Medium (2) | High (3) | **6** | High |
|
|
|
| **TECH-002** | AASM gem integration issues or configuration errors | TECH | Low (1) | Medium (2) | **2** | Low |
|
|
|
| **SEC-001** | Feature flag bypass - FloDoc routes not properly protected | SEC | Medium (2) | High (3) | **6** | High |
|
|
|
| **SEC-002** | Email validation gaps on sponsor_email/student_email | SEC | Low (1) | Medium (2) | **2** | Low |
|
|
|
| **PERF-001** | N+1 queries on model associations (institution→cohorts→enrollments) | PERF | High (3) | Medium (2) | **6** | High |
|
|
|
| **PERF-002** | Missing database indexes on frequently queried columns | PERF | Medium (2) | Medium (2) | **4** | Medium |
|
|
|
| **DATA-001** | Foreign key constraint violations with existing tables | DATA | Medium (2) | High (3) | **6** | High |
|
|
|
| **DATA-002** | JSONB field validation failures (required_student_uploads, cohort_metadata) | DATA | Low (1) | Medium (2) | **2** | Low |
|
|
|
| **DATA-003** | Unique constraint violations (cohort_enrollments.submission_id) | DATA | Low (1) | High (3) | **3** | Low |
|
|
|
| **BUS-001** | State machine logic doesn't match business workflow | BUS | Medium (2) | High (3) | **6** | High |
|
|
|
| **OPS-001** | Feature flag seed data missing or incorrect | OPS | Low (1) | Low (1) | **1** | Minimal |
|
|
|
| **OPS-002** | Test coverage below 80% target | OPS | Medium (2) | Medium (2) | **4** | Medium |
|
|
|
|
|
|
---
|
|
|
|
|
|
## Critical Risks (Score 6)
|
|
|
|
|
|
### 1. TECH-001: State Machine Complexity
|
|
|
**Score: 6 (High)**
|
|
|
**Probability**: Medium - Complex state transitions with 7 states
|
|
|
**Impact**: High - Incorrect workflow could block business operations
|
|
|
|
|
|
**Description**: The Cohort model implements a 7-state machine (draft → tp_signed → students_completed → sponsor_completed → finalized → active → completed) with complex transition rules. Missing guard clauses or incorrect transitions could cause data integrity issues.
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Implement comprehensive state transition tests for all valid/invalid transitions
|
|
|
- Add guard clauses for state transitions (e.g., cannot skip steps)
|
|
|
- Document state machine diagram in code comments
|
|
|
- Test edge cases: concurrent state changes, rollback scenarios
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Unit tests for all state transition events (10+ scenarios)
|
|
|
- Integration tests for complete workflow (draft → completed)
|
|
|
- Edge case: Invalid transitions should raise errors
|
|
|
- Concurrency tests for simultaneous state changes
|
|
|
|
|
|
---
|
|
|
|
|
|
### 2. SEC-001: Feature Flag Bypass
|
|
|
**Score: 6 (High)**
|
|
|
**Probability**: Medium - Missing before_action in controllers
|
|
|
**Impact**: High - FloDoc functionality exposed prematurely
|
|
|
|
|
|
**Description**: FloDoc routes must be protected by feature flag checks. Missing protection could expose functionality before it's ready for production.
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Implement FeatureFlagCheck concern with require_feature helper
|
|
|
- Add controller specs that verify feature flag protection
|
|
|
- Test both enabled and disabled states
|
|
|
- Create integration test for full request flow
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Controller specs with feature flag enabled/disabled
|
|
|
- Request specs verifying 404/403 when flag disabled
|
|
|
- Test feature flag toggle functionality
|
|
|
- Verify all FloDoc routes are protected
|
|
|
|
|
|
---
|
|
|
|
|
|
### 3. PERF-001: N+1 Query Issues
|
|
|
**Score: 6 (High)**
|
|
|
**Probability**: High - Common issue with nested associations
|
|
|
**Impact**: Medium - Performance degradation with 1000+ records
|
|
|
|
|
|
**Description**: Models have nested associations (institution→cohorts→enrollments). Without proper eager loading, queries could cause N+1 performance issues.
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Use `includes()` or `eager_load()` for all association queries
|
|
|
- Add performance tests with 1000+ test records
|
|
|
- Use Bullet gem or similar to detect N+1 queries
|
|
|
- Verify with EXPLAIN queries
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Performance tests with large datasets (1000+ records)
|
|
|
- Query optimization verification
|
|
|
- Association loading tests
|
|
|
- EXPLAIN query analysis for slow queries
|
|
|
|
|
|
---
|
|
|
|
|
|
### 4. DATA-001: Foreign Key Constraint Violations
|
|
|
**Score: 6 (High)**
|
|
|
**Probability**: Medium - Referencing non-existent template/submission IDs
|
|
|
**Impact**: High - Data integrity issues, failed saves
|
|
|
|
|
|
**Description**: Models reference existing DocuSeal tables (templates, submissions). Foreign key constraints could prevent saves if referenced records don't exist.
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Validate foreign key existence before save
|
|
|
- Create test helpers for dependent records
|
|
|
- Add database-level foreign key constraints
|
|
|
- Test rollback scenarios
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Integration tests with real foreign key references
|
|
|
- Test data integrity with missing references
|
|
|
- Verify FK constraints prevent orphaned records
|
|
|
- Test cascading delete/soft delete behavior
|
|
|
|
|
|
---
|
|
|
|
|
|
### 5. BUS-001: State Machine Logic Mismatch
|
|
|
**Score: 6 (High)**
|
|
|
**Probability**: Medium - Business requirements vs implementation
|
|
|
**Impact**: High - Workflow doesn't match business needs
|
|
|
|
|
|
**Description**: State machine implementation must match PRD business requirements. Mismatch could cause workflow failures.
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Validate state machine against PRD requirements
|
|
|
- Get business stakeholder review of state transitions
|
|
|
- Document business rules for each state
|
|
|
- Add acceptance criteria tests for state transitions
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Business requirement validation tests
|
|
|
- State transition approval tests
|
|
|
- Workflow completion tests
|
|
|
- PRD requirement traceability tests
|
|
|
|
|
|
---
|
|
|
|
|
|
## Medium Risks (Score 4)
|
|
|
|
|
|
### 6. PERF-002: Missing Database Indexes
|
|
|
**Score: 4 (Medium)**
|
|
|
**Probability**: Medium - Indexes not added on queried columns
|
|
|
**Impact**: Medium - Query performance degradation
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Add indexes on all foreign keys
|
|
|
- Add indexes on frequently queried columns (status, email)
|
|
|
- Verify index usage with EXPLAIN queries
|
|
|
- Test query performance with large datasets
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Database migration specs for index creation
|
|
|
- EXPLAIN query analysis
|
|
|
- Performance tests with 1000+ records
|
|
|
|
|
|
---
|
|
|
|
|
|
### 7. OPS-002: Test Coverage Below 80%
|
|
|
**Score: 4 (Medium)**
|
|
|
**Probability**: Medium - Insufficient test coverage
|
|
|
**Impact**: Medium - Quality issues, bugs in production
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Calculate test coverage after implementation
|
|
|
- Add missing test scenarios
|
|
|
- Use coverage tools (SimpleCov, RCov)
|
|
|
- Ensure >80% coverage requirement is met
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Unit test coverage for all models
|
|
|
- Integration test coverage for workflows
|
|
|
- Feature flag protection tests
|
|
|
- State machine transition tests
|
|
|
|
|
|
---
|
|
|
|
|
|
## Low Risks (Score 1-3)
|
|
|
|
|
|
### 8. DATA-003: Unique Constraint Violations
|
|
|
**Score: 3 (Low)**
|
|
|
**Probability**: Low - Duplicate submission_id
|
|
|
**Impact**: High - Data integrity issues
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Add unique constraint on cohort_enrollments.submission_id
|
|
|
- Test duplicate submission handling
|
|
|
- Verify constraint prevents duplicates
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Unit tests for unique constraint
|
|
|
- Integration tests for duplicate prevention
|
|
|
- Error handling for constraint violations
|
|
|
|
|
|
---
|
|
|
|
|
|
### 9. SEC-002: Email Validation Gaps
|
|
|
**Score: 2 (Low)**
|
|
|
**Probability**: Low - Missing format validation
|
|
|
**Impact**: Medium - Invalid email data
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Add email format validation to all email fields
|
|
|
- Test valid/invalid email formats
|
|
|
- Verify validation errors are raised
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Unit tests for email validation
|
|
|
- Integration tests for email format checking
|
|
|
|
|
|
---
|
|
|
|
|
|
### 10. DATA-002: JSONB Field Validation Failures
|
|
|
**Score: 2 (Low)**
|
|
|
**Probability**: Low - Invalid JSON data
|
|
|
**Impact**: Medium - Data corruption
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Add JSON schema validation for complex fields
|
|
|
- Test valid/invalid JSON data
|
|
|
- Verify validation errors are raised
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Unit tests for JSONB field validation
|
|
|
- Integration tests for data integrity
|
|
|
|
|
|
---
|
|
|
|
|
|
### 11. TECH-002: AASM Gem Integration Issues
|
|
|
**Score: 2 (Low)**
|
|
|
**Probability**: Low - Gem configuration errors
|
|
|
**Impact**: Medium - State machine not working
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Verify AASM gem installation and configuration
|
|
|
- Test state machine initialization
|
|
|
- Verify event callbacks work correctly
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Unit tests for AASM configuration
|
|
|
- Integration tests for state machine functionality
|
|
|
|
|
|
---
|
|
|
|
|
|
## Minimal Risks (Score 1)
|
|
|
|
|
|
### 12. OPS-001: Feature Flag Seed Data Missing
|
|
|
**Score: 1 (Minimal)**
|
|
|
**Probability**: Low - Seed data not created
|
|
|
**Impact**: Low - Feature flag not available
|
|
|
|
|
|
**Mitigation Strategy**:
|
|
|
- Create seed data for feature flags
|
|
|
- Test seed data creation
|
|
|
- Verify feature flags exist in database
|
|
|
|
|
|
**Testing Focus**:
|
|
|
- Seed data tests
|
|
|
- Feature flag availability tests
|
|
|
|
|
|
---
|
|
|
|
|
|
## Risk-Based Testing Strategy
|
|
|
|
|
|
### Priority 1: Critical Risk Tests (Score 6+)
|
|
|
1. **State Machine Tests** - All 7 states, all transitions (TECH-001, BUS-001)
|
|
|
2. **Feature Flag Protection Tests** - Controller/request level (SEC-001)
|
|
|
3. **Foreign Key Constraint Tests** - Integration with existing tables (DATA-001)
|
|
|
4. **N+1 Query Detection Tests** - Performance with 1000+ records (PERF-001)
|
|
|
5. **Business Workflow Validation Tests** - State transitions match PRD (BUS-001)
|
|
|
|
|
|
### Priority 2: High Risk Tests (Score 4)
|
|
|
1. **Database Index Tests** - Verify indexes on queried columns (PERF-002)
|
|
|
2. **Test Coverage Verification** - >80% coverage requirement (OPS-002)
|
|
|
|
|
|
### Priority 3: Medium/Low Risk Tests (Score 1-3)
|
|
|
1. **Email Validation Tests** - Format validation on all email fields (SEC-002)
|
|
|
2. **JSONB Field Tests** - Validation of complex fields (DATA-002)
|
|
|
3. **Unique Constraint Tests** - submission_id uniqueness (DATA-003)
|
|
|
4. **Feature Flag Seed Tests** - Default flags present (OPS-001)
|
|
|
|
|
|
---
|
|
|
|
|
|
## Risk Acceptance Criteria
|
|
|
|
|
|
### Must Fix Before Production
|
|
|
- All critical risks (score 6) must be mitigated
|
|
|
- State machine must pass all transition tests
|
|
|
- Feature flag protection must be verified
|
|
|
- Foreign key constraints must be tested
|
|
|
- Test coverage must exceed 80%
|
|
|
|
|
|
### Can Deploy with Mitigation
|
|
|
- Medium risks (score 4) with compensating controls
|
|
|
- Low risks (score 2-3) with monitoring in place
|
|
|
|
|
|
### Accepted Risks
|
|
|
- Minimal risks (score 1) can be accepted with documentation
|
|
|
- Performance optimization can be deferred if within NFR limits
|
|
|
|
|
|
---
|
|
|
|
|
|
## Monitoring Requirements
|
|
|
|
|
|
Post-deployment monitoring for:
|
|
|
- **Performance metrics** - Query times with 1000+ records
|
|
|
- **Error rates** - State machine transition failures
|
|
|
- **Feature flag usage** - Toggle frequency and impact
|
|
|
- **Data integrity** - Foreign key constraint violations
|
|
|
|
|
|
---
|
|
|
|
|
|
## Risk Review Triggers
|
|
|
|
|
|
Review and update risk profile when:
|
|
|
- State machine requirements change
|
|
|
- New associations added to models
|
|
|
- Feature flag system modified
|
|
|
- Performance issues reported in production
|
|
|
- Business workflow changes
|
|
|
|
|
|
---
|
|
|
|
|
|
## Gate YAML Block Output
|
|
|
|
|
|
```yaml
|
|
|
risk_summary:
|
|
|
totals:
|
|
|
critical: 0 # score 9
|
|
|
high: 5 # score 6
|
|
|
medium: 2 # score 4
|
|
|
low: 4 # score 2-3
|
|
|
minimal: 1 # score 1
|
|
|
highest:
|
|
|
id: TECH-001
|
|
|
score: 6
|
|
|
title: 'State machine complexity - 7 states with complex transitions'
|
|
|
recommendations:
|
|
|
must_fix:
|
|
|
- 'Implement comprehensive state transition tests for all valid/invalid transitions'
|
|
|
- 'Add FeatureFlagCheck concern with require_feature helper in controllers'
|
|
|
- 'Use includes() or eager_load() for all association queries to prevent N+1'
|
|
|
- 'Validate foreign key existence before save with test helpers'
|
|
|
- 'Verify state machine logic matches PRD business requirements'
|
|
|
monitor:
|
|
|
- 'Monitor query performance with 1000+ records post-deployment'
|
|
|
- 'Track feature flag toggle frequency and errors'
|
|
|
- 'Alert on state machine transition failures'
|
|
|
- 'Monitor foreign key constraint violations'
|
|
|
```
|
|
|
|
|
|
---
|
|
|
|
|
|
## Key Principles Applied
|
|
|
|
|
|
✅ **Risk-Based Testing** - Focused on high-impact areas
|
|
|
✅ **Probability × Impact** - Systematic scoring (6 high, 2 medium, 5 low)
|
|
|
✅ **Actionable Mitigation** - Specific testing strategies for each risk
|
|
|
✅ **Gate-Ready Output** - YAML format for quality gate integration
|
|
|
✅ **Business Alignment** - State machine validation against PRD
|
|
|
|
|
|
---
|
|
|
|
|
|
**Risk Score: 42/100** (Lower is better - 100 = no risk)
|
|
|
|
|
|
**Recommendation:** Address all 5 high-risk items before implementation. The state machine complexity and feature flag protection are the most critical risks that could block production deployment.
|