Skip to main content

CI/CD dimension

1 - Reactive

Description​

  • Code integration and software deployment are done mostly manually, often leading to inconsistencies, errors, or decreased productivity.
  • Frequent integration issues (e.g., "it works on my machine").
  • CI/CD practices are either non-existent or inconsistently applied, leading to inefficiencies, delays, and quality issues.
  • Manual build and deployment processes.
  • Lack of automated checks at the pull request level.
  • Limited visibility into build or deployment status.

2 - Managed

Description​

  • Basic working CI/CD pipeline and practices are in place.
  • The CI/CD agents are unstable, requiring regular maintenance and often demanding manual intervention to ensure continued functionality.
  • Slow feedback from CI/CD processes, long waiting times for build and deploy.
  • Automated builds are established but may not occur on every change.
  • Failures are hard to trace back, lacking detailed logging or feedback.
  • Mandatory checks at PR level:
    • Require a smoke test suite to pass before any code merge.
    • Run tests at the PR level and report coverage and results.

3 - Defined

Description​

  • The CI/CD process is well-documented and consistently followed.
  • Build and deployment failures are systematically addressed; root causes are identified, documented, and fixed.
  • High availability of CI/CD agents; downtimes are rare and scheduled.
  • Detailed logs are available, making it easier to trace failures.
  • Every integration is verified by an automated build and automated tests, ensuring early detection of integration errors.
  • All test types/levels integrated into the CI/CD pipeline.
  • Builds are triggered automatically on every PR, ensuring code is consistently integrated.
  • All dependencies are managed using a package manager and applications' dependencies are locked.
  • Deployments are semi-automated and may still require manual oversight.

4 - Measured

Description​

  • CI/CD pipelines are refined based on feedback and metrics.
  • Near-constant availability; any downtime is swiftly addressed.
  • Advanced tools and monitoring in place, allowing for real-time debugging and tracing.
  • Builds and deployments are optimized for speed, utilizing parallel processes and efficient resources.
  • Builds are at least partially reproducible.
  • Performance and other quality attributes are monitored and measured, feeding back into the CI/CD process.
  • Deployment to environments like staging is predictable and consistent.
  • Metrics** and feedback loops are in place to monitor and improve the CI/CD pipeline's efficiency and reliability.
  • Full integration across all tools in the CI/CD pipeline, enabling a completely automated flow.

5 - Optimized

Description​

  • Failures are extremely rare; focus has shifted from stability to continuous optimization.
  • Redundant systems ensure constant availability of CI/CD agents and tools.
  • Continuous feedback allows for real-time optimizations; feedback loop is as short as possible.
  • CI/CD pipeline, test automation, and test coverage are sophisticated, efficient, and continuously reviewed and improved based on regular review and feedback.
  • Ability to deploy entirely automatically, including production environments.
  • There's a heavy emphasis on feedback loops, with real-time monitoring, logging, and analytics in place to inform the development teams of issues or areas for improvement.
  • Builds are fully reproducible.

** some potential core metrics: Cycle Time, Build Failure Rate, Defect Leakage, CI/CD Failure Rate, Master Pipeline Stability, Deploy Frequency, Code Coverage, etc

Guiding questions​

  1. Integration Efficiency: How effectively and seamlessly does our current CI/CD pipeline integrate new code changes, and what measures do we have in place to ensure this process is both efficient and error-free?
  2. Automated Testing Coverage: What percentage of your codebase is covered by automated tests, and how do you ensure the quality and effectiveness of these tests?
  3. Automation Level: To what extent have we automated key processes in our CI/CD pipeline, including testing, integration, and deployment, and what opportunities exist for further automation?
  4. Build Automation: To what extent are your build processes automated, and what manual steps still exist?
  5. Deployment Strategy: What strategies are we using for deployment (e.g., blue-green, canary, rolling updates), and how do they align with our overall operational goals and risk management?
  6. CI/CD Pipeline Efficiency: What are the most time-consuming stages in your CI/CD pipeline, and what steps have you taken to optimize them?
  7. Monitoring and Feedback Mechanisms: How do you monitor deployments, and how quickly can you respond to and rectify a failed deployment?
  8. Collaboration and Communication: How well do our CI/CD practices facilitate collaboration and communication within the team and with other stakeholders, and what steps can we take to enhance this?
  9. Monitoring and Metrics: "What metrics and monitoring tools are we currently using to track the performance and effectiveness of our CI/CD practices, and how do these metrics inform our continuous improvement efforts?