If you build a workplace where every team member feels happy, then you can have more productivity of high-quality codes. It would empower them to quickly see the benefits and results of their work which is the ulterior motive of any software development team. In practice it is challenging and requires consistent assessment and reassessment of technology used, processes followed which would make your product and your team unique. You must have an operation infrastructure which is friendly, communicative and sharing regarding responsibilities and duties. With the absence of a perfect working environment, you are bound to have faulty codes resulting in tech debt and early death of your company.
Points To Focus On
You must think that strategic improvement of working environment is not just firefighting and need some specific points to focus on to have the perfect working environment. If you implement an on-call rotation, it will help the engineers to see precisely all those unexpected things that had happened during the production of code. If you do not do it for your entire engineering team, you will lose on the precious training as well as growth in your infrastructure and your development team. You must not limit this on-call rotation to the backend engineers only to have the desired effect.
The New Recruits
If you want to grow the in-house knowledge and increase the rotation size, train any engineer who is interested in being on call without considering the background of the person. The on call boot camp must consist of architectural presentations, hands-on deployment, business hour pager duty so that you can give your engineers responsibility of on call and have a quick escalation path. On calls can be used as the best motivators to bring any change in the infrastructure. It is not possible for any engineer to know all the parts of a product and therefore you must recruit new people and train them all from time to time.
Alerting Is Critical
When something goes wrong, it is critical to alert your engineers who are being constantly interrupted and be in firefighting mode always. When there is any broken piece in your infrastructure, it is a real problem and may make it difficult for you to prioritize the work which needs to be done. Track the alert noises and write scripts using proper API but ultimately settle with a simple document which would help in prioritizing.
Collective Ownership Helps
When you build a team of happy engineers, it results in collective ownership with increased trust among each other. It results in improvement in coding practices and in building good codes with bold ideas so that it flourishes organically. Lastly, to have a better working environment, you have to make your team understand tech debt and the reasons for its existence. You must train them not to blame others for it but takes the responsibility to repay it at the earliest because you cannot ignore tech debt for a long time and allow accumulating. Just like you pay off your financial debt with best consolidation loans, you must also pay off tech debt remain in the race.