In Might 2026, Pope Leo XIV launched his first encyclical, “Magnifica humanitas,” a textual content dedicated to safeguarding the human individual within the time of synthetic intelligence (Leo XIV, 2026). He selected to current it within the Vatican alongside an surprising visitor, Chris Olah, a co-founder of Anthropic and one of many individuals constructing the frontier programs the doc examines.
Olah used the event to say one thing a know-how govt just isn’t anticipated to say. Each frontier lab, his personal included, operates inside incentives that may battle with doing the proper factor, and no quantity of fine intention escapes them. For that purpose, he argued, the world wants individuals exterior these incentives, governments and civil society amongst them, to set the bounds that the labs can’t be trusted to set for themselves. “We want knowledgeable critics who will inform the labs after we are failing,” he stated. “We want ethical voices that the incentives can’t bend” (Olah, 2026).
When a co-founder of a number one AI firm travels to Rome to ask the skin world to control his personal business, the acquainted declare that regulation is the enemy of progress has misplaced its most credible narrator.
The decisive query for software program over the following decade just isn’t which mannequin your workforce adopts. It’s whether or not the individuals who construct software program keep in authority over it as synthetic intelligence spreads by the work, and well-designed regulation is how Europe is about to make the human-first reply binding moderately than aspirational.
The dominant business story says the other, treating the European Union’s regulatory programme as a brake on ambition that cedes the long run to corporations working underneath looser guidelines. Olah’s testimony factors the opposite approach. When each business incentive pushes towards letting AI quietly resolve extra about how engineers work and the way they’re judged, regulation is the one pressure presently sturdy sufficient to carry the road on a human-first path, as a result of that path isn’t the most affordable one within the brief time period, and the market is not going to defend it unaided.
This isn’t an argument towards measuring software program work, and it isn’t a grievance about developer-experience tooling. Good measurement, performed on the stage of the workforce and used to enhance how individuals work, is a part of a human-first path. The priority is narrower and sharper: the rising class of programs that use AI to guage, rank, and resolve about particular person engineers, with the human diminished to somebody who approves a quantity the machine produced. The AI Act attracts a line exactly there, and the road it attracts is the clearest assertion in present legislation of what human-first software program really calls for.
What a human-first path really requires
Begin with what the dedication means in observe, as a result of the phrase is simple to wave at and onerous to honour. The Copenhagen Manifesto units out the take a look at instantly: generative AI in software program engineering have to be evaluated first by what individuals want, and solely then by what the know-how can do (Russo et al., 2024). A device earns its place not by uncooked velocity however by its impact on the wellbeing, studying, and autonomy of the engineers who use it. That ordering, individuals first and know-how second, is the entire of the human-first concept.
The proof on adoption explains why the ordering just isn’t sentimental. Russo’s research of generative AI adoption throughout software program engineering, constructed from structured interviews with 100 software program engineers and a mixed-methods design, discovered that uptake is ruled by particular person, technological, and social elements appearing collectively moderately than by any property of the device in isolation (Russo, 2024). The ensuing Human-AI Collaboration and Adaptation Framework treats adoption as one thing individuals do inside a context, weighing whether or not a device suits how they already work and who they perceive themselves to be. A system that designs the individual out of that context, treating the engineer as an enter to be optimised moderately than knowledgeable to be supported, is working towards the one mechanism by which adoption succeeds.
A human-first path, then, just isn’t a slogan about being good to engineers however a design self-discipline. It retains a reliable individual in real authority over consequential selections, it preserves the autonomy that lets professional judgment function, and it refuses to let effectivity erase accountability. The query for European software program is whether or not something ensures groups really construct this manner when the cheaper various is at all times obtainable. Left to incentives alone, the reply is not any.
The place the legislation attracts the road
That is the place the AI Act turns into concrete moderately than summary. Annex III of Regulation (EU) 2024/1689 lists the makes use of the legislation treats as high-risk, and level 4(b) names AI meant to guage and monitor the efficiency and behavior of individuals in work relationships, to allocate duties primarily based on particular person behaviour or traits, or to tell promotion and termination (European Fee, 2024). A system that scores engineers on responsiveness, reliability, or output, and routes these scores towards a call about pay, rating, or continued employment, falls inside that definition and inherits the total obligation set that comes with high-risk standing.
Two design selections within the legislation make this difficult to evade, and each specific the human-first precept in authorized type.
The primary issues profiling. Some Annex III programs can argue out of high-risk standing by displaying they carry out solely a slim preparatory operate, however the Fee’s draft classification pointers, printed for session in Might 2026, take away that escape for programs that profile an individual underneath the Basic Information Safety Regulation (European Fee, 2026). Evaluating a developer’s reliability and behavior is profiling by definition, so the exemption that rescues different office instruments doesn’t attain this use. The legislation treats judging an individual as categorically extra severe than processing a factor.
The second alternative addresses the snug perception {that a} human reviewer makes an automatic scoring system protected. The rules reject that reasoning at its root.
“Since human involvement can’t change the aim and space during which a system is meant for use, it has no impact on the classification of the system as high-risk underneath Article 6(2).” (European Fee, 2026, para. 70)
The implication is the human-first precept acknowledged as a rule. A human positioned within the loop is there to train actual authority over the end result, which is an obligation the legislation imposes underneath Article 14, and never a token whose presence lets the classification disappear. A supervisor who rubber-stamps an algorithmic rating has not made the system protected; he has demonstrated why oversight was required to start with. The rules prolong the identical logic to structure, assessing a pipeline cut up throughout separate telemetry, scoring, and reporting parts as a single system the place its mixed outputs form a call about an individual (European Fee, 2026, para. 75). A modular design can’t launder an evaluative objective into one thing the legislation overlooks.
Learn rigorously, the rule is permissive about precisely the practices a human-first workforce would already settle for. Programs that floor impartial, goal elements comparable to availability or location keep exterior the high-risk class. Analytics that mixture to the workforce stage, with out rating identifiable people and with out feeding pay, promotion, or process allocation, additionally keep exterior it. The boundary the regulation attracts is the boundary between measuring work to assist a workforce enhance and utilizing AI to go judgment on an individual.
Why the market is not going to select this by itself
If the human-first path have been additionally the most affordable path, no regulation could be wanted. It isn’t, and that hole is the sincere case for the legislation.
Particular person scoring is engaging exactly as a result of it’s low cost, legible to executives, and simple to automate. A dashboard that ranks engineers on commits or closed tickets produces a quantity a pacesetter can act on with out understanding the work behind it. The difficulty is that the quantity is a poor measure and an energetic hazard. Software program worth is a joint product of individuals, code, and course of underneath uncertainty, and decomposing that system into per-engineer counters discards the interplay results that designate a lot of the variance in outcomes. Worse, a metric connected to analysis turns into a goal inside a couple of quarter, and from that time it measures the behaviour it rewards moderately than the work it was meant to trace. A budget path can be the trail that quietly degrades the factor it claims to handle.
So the market, left alone, drifts towards automation-first measurement as a result of the prices of doing so are deferred and diffuse, whereas the financial savings are fast and visual. That is the failure Olah named from the Vatican, an incentive construction that no single agency escapes by good intention alone. A binding ground adjustments that calculation by making particular person developer analysis a high-risk, accountable act, the AI Act raises the price of a budget path to the purpose the place the human-first path competes on equal phrases. That is what regulation is for in a website the place the externalities fall on individuals who don’t have any say within the tooling imposed on them.
The objection that this burdens European corporations towards less-regulated rivals errors a legal responsibility for a bonus. A agency that builds AI to handle its engineers with out accountability has not discovered a shortcut; it has accrued publicity that the revised Product Legal responsibility Directive, in pressure throughout the Union from December 2026, will finally worth (European Parliament and Council, 2024). The constraint that appears like a price is the self-discipline that retains the individuals who construct software program sovereign over their very own work, and that sovereignty is the asset European software program ought to most need to defend.
The compliance clock and the broader stack
Welcoming the course of the legislation doesn’t imply the preparation is trivial, and the timing deserves precision as a result of confusion about it’s the most cited objection. Excessive-risk obligations underneath Annex III have been initially set to use from 2 August 2026. A political settlement on the Digital Omnibus, reached in Might 2026, would defer them to 2 December 2027, however that deferral takes authorized impact solely as soon as it’s printed within the Official Journal, which had not occurred on the time of writing. A disciplined organisation builds to the sooner date and re-baselines if the deferral is enacted. The AI literacy responsibility underneath Article 4 and the penalty regime are already reside, so the readiness work just isn’t an issue for some distant 12 months.
Developer-evaluation programs additionally sit inside a wider regulatory stack {that a} European software program organisation can’t deal with in isolation. The Digital Operational Resilience Act applies to corporations serving monetary entities, the Cyber Resilience Act brings vulnerability reporting from September 2026 and full obligations from December 2027, and NIS2 governs safe growth and incident reporting. Every regime asks for a similar underlying factor, which is proof: classification data, technical documentation, computerized logs retained for not less than six months, oversight reviews, and influence assessments. An organisation that builds this proof functionality as soon as, across the human-first dedication, can serve a number of regimes from it moderately than treating every as a separate scramble.
A readiness audit for European engineering groups
Run this towards any AI system your organisation makes use of to watch, measure, or handle builders. Every merchandise maps to a selected obligation, and every hole is a documented threat.
1. Write a classification document for each developer-facing AI system, stating its meant objective, the individuals it touches, and whether or not its output feeds a call about an identifiable particular person. The document is itself a required artefact.
2. Resolve whether or not the system profiles people underneath the GDPR definition. If it evaluates reliability, behaviour, or efficiency, deal with it as profiling, and deal with the preparatory-task exemption as unavailable.
3. Hint the place every output goes. If something routes towards pay, promotion, process allocation, or termination, deal with the system as high-risk, nevertheless many individuals overview it.
4. Verify that computerized occasion logging is enabled and that logs are retained for not less than six months inside your personal perimeter, not solely in a vendor’s cloud.
5. Doc your human-oversight design underneath Article 14: who oversees the system, what competence they maintain, and the way their overrides are recorded.
6. Confirm that employees’ representatives and affected builders have been knowledgeable earlier than the system went into use, as Article 26(7) requires.
7. Produce a knowledge safety influence evaluation for systematic worker monitoring, and put together a fundamental-rights influence evaluation the place the deployer scope applies.
8. Report dated AI-literacy coaching for the employees and contractors who function or oversee the system, since that obligation is already in pressure.
9. Preserve a per-obligation effective-date register, defaulting high-risk dates to 2 August 2026 till any Omnibus deferral is printed within the Official Journal.
10. Set retention guidelines intentionally: logs for not less than six months, core technical documentation for ten years, with data-minimisation overrides the place employee-monitoring proportionality requires a lighter footprint.
Subsequent strikes
For the builder
Discover out what each AI device in your workflow data about you and the place these data journey. You now have a recognised stake within the reply.
When a device’s output might form a call about you, ask in writing for overview by a named individual with the authority to overturn it, and deal with something much less because the rubber stamp the legislation was written to forestall.
Hold an everyday window the place you write onerous code unaided. Authority over your personal craft is a functionality you keep by observe, not a standing the organisation grants you.
When a compliance coverage is unclear to you, write down your interpretation and flow into it, since forcing the anomaly into the open is itself an act of governance.
For the supervisor
Decline to produce per-engineer rankings upward, and supply team-level proof throughout a number of dimensions as an alternative. The regulation now offers you authorized cowl for a place the proof already helps.
Translate any central compliance body into a brief working notice your workforce can act on inside two weeks of receiving it. When you can’t, the coverage just isn’t implementable, and that discovering is value sending again upward.
Full the worker-notification step earlier than deploying any monitoring or measurement system, treating it as a trust-building act moderately than a formality.
Defend overview and considering time because the definition of performed shifts underneath AI help, in order that measured velocity doesn’t eat the cognitive house on which common sense relies upon.
For the roadmap proprietor
Deal with developer-measurement AI as a high-risk system in each build-or-buy determination, and require distributors to point out how their product generates the proof the legislation calls for inside your perimeter moderately than in a dashboard you can not audit.
Rise up a per-obligation effective-date register now, default it to the conservative dates, and assign an proprietor to re-baseline it the day any deferral publishes.
Fund the compliance proof as a product functionality moderately than an afterthought, for the reason that classification data, logs, and oversight reviews are the identical artefacts that may defend you towards a future product-liability declare.
Audit your engineering working mannequin for practices that strip autonomy, together with centralised approval bottlenecks, mandated AI tooling with out an opt-out, and metric-driven efficiency overview, every of which now carries regulatory weight on prime of its cultural value.
Closing thought
The regulation reframes a query many organisations have most well-liked to keep away from. The model value asking just isn’t learn how to measure builders extra exactly, however which of your present practices would survive being categorized as a call about an individual, and what that claims about people who wouldn’t. The groups that learn this as paperwork will produce paperwork. The groups that learn it as an opportunity to intentionally decide to a human-first path will find yourself with each compliance and higher engineering.
Daniel Russo, Ph.D., is a Professor of Software program Engineering whose analysis examines the intersection of human cognition and synthetic intelligence. By means of “Software program Insights,” he interprets empirical analysis into actionable steerage for software program practitioners and organizations.
If this problem surfaces an issue your organisation has been attempting to call, I work with engineering leaders to diagnose precisely that sort of problem, utilizing the identical strategies behind the analysis you simply learn. No frameworks. No opinion with out proof.
danielrusso.org/advisory (Opens in a new window)
References
European Fee. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised guidelines on synthetic intelligence (Synthetic Intelligence Act). Official Journal of the European Union.
European Fee. (2026). Draft pointers on the classification of high-risk AI programs underneath Regulation (EU) 2024/1689 (focused session, Might 2026).
European Parliament and Council. (2016). Regulation (EU) 2016/679 (Basic Information Safety Regulation). Official Journal of the European Union.
European Parliament and Council. (2024). Directive (EU) 2024/2853 on legal responsibility for faulty merchandise. Official Journal of the European Union.
Leo XIV. (2026). Magnifica humanitas: On safeguarding the human individual within the time of synthetic intelligence [Encyclical letter]. Holy See.
Olah, C. (2026). Remarks on Pope Leo XIV’s encyclical “Magnifica humanitas.” Anthropic.
Russo, D. (2024). Navigating the complexity of generative AI adoption in software program engineering. ACM Transactions on Software program Engineering and Methodology, 33(5), Article 135.
Russo, D., Baltes, S., van Berkel, N., Avgeriou, P., Calefato, F., Cabrero-Daniel, B., et al. (2024). Generative AI in software program engineering have to be human-centered: The Copenhagen Manifesto. Journal of Programs and Software program, 216, 112115.