The safe use of AI in software development is not just a technical concern but a question of human responsibility. As AI systems become better at generating, testing, and optimising code, the developer's identity itself is evolving. For decades, the act of writing code has been defined by the capacity to translate logic into structure, to create intentional systems from abstract ideas. Now, this foundation is shifting. The developer is no longer the exclusive author of code but increasingly also the evaluator of what machines propose.
This transition toward greater curatorial responsibility defines the central challenge of safe AI use. It requires a realignment of what competence means, expanding from producing code efficiently to understanding it deeply, regardless of whether it was written by hand or generated by AI. Safe use depends not only on technical skill alone, but on judgment; namely, on the ability to question assumptions, interpret intent, and identify risks that no model can anticipate. Importantly, this curatorial dimension complements rather than replaces the creative and technical work of designing, reasoning about, and implementing systems. Developers remain builders, but they are builders who must now also thoughtfully integrate AI tools into their craft.
However, this shift also carries emotional and cultural weight that needs to be acknowledged. For many, coding is not just work but an identity. As AI begins to share that creative space, developers face the dual task of maintaining both the integrity of their systems and the meaning of their role within them.
The sections that follow examine this transformation through several complementary lenses. "What does 'developer' mean Now?" explores how AI-assisted coding redefines authorship, comprehension, and responsibility, drawing parallels with similar transitions in other creative disciplines. "The curator's challenge" looks at the cognitive and behavioural biases that influence human oversight of AI-generated code, highlighting how perceived efficiency can mask genuine risks. "Governance that enables the curator role" outlines how regulatory frameworks such as the EU AI Act and the GDPR formalise this new responsibility, placing human judgment at the centre of safe AI practice. And, finally, "Supporting the curator role in practice" reflects on how Nordic professional culture, rooted in trust, collaboration, and accountability, offers a sustainable blueprint for integrating AI safely into development work.
Ultimately, the argument is simple yet profound. Safe AI use is not about restricting automation but about cultivating discernment. Developers remain the keepers of meaning, ethics, and security in a landscape increasingly shaped by machine fluency. Their role does not diminish; it expands. True expertise now lies in both creation and comprehension – maintaining the craftsmanship of designing and building systems while ensuring that what machines produce remains explainable, lawful, and humanly accountable.
What does "developer" mean now?
The key challenge in discussing the safe use of AI in software development is not the technology itself but the evolving identity of the developer. For most of the history of computing, the developer's role has been defined by authorship and skills. Writing code requires minds that can express themselves logically and in a structured, intentional manner. AI-assisted coding adds a new dimension to this relationship. The developer remains an author but increasingly also becomes the evaluator, curator, and editor of machine-generated material alongside their own creations.
This shift is also evident in other creative domains. In journalism, for example, automated text generation has encouraged writers to act as editors, ensuring factual integrity and stylistic coherence, rather than merely producing sentences. In the same way, developers must now read code not only for its functionality but also for its meaning, security, and compliance, whether that code originates from their own design work or from AI suggestions. The safe use of AI, therefore, requires adding comprehension metrics to production metrics, rather than replacing one with the other.
The developer must be able to answer three questions that the machine cannot:
- Why does this code behave as it does?
- What assumptions does it embed?
- What risks does it introduce?
These are questions of judgment, not generation. Yet, judgment itself depends on deep technical expertise such as the ability to design systems, reason about trade-offs, and implement solutions that AI cannot yet produce autonomously. The curatorial role enhances rather than diminishes the need for these foundational skills.
The METR¹ findings reinforce this need. When experienced developers misjudge the true impact of AI on their performance, believing themselves to be faster while, in fact, being slower, it demonstrates how easily the appearance of fluency can hide the depth of understanding required for effective oversight (METR, 2025). Effective curation, therefore, demands depth of comprehension rather than breadth of output.
The transformation is also psychological. As Annie Vella (a lifelong computer enthusiast with two decades of experience in software engineering and technical leadership) observes in her blog, The Software Engineering Identity Crisis (2025)², many developers experience the transition from creator to evaluator as a loss: "Many of us don't just write code – we love writing code. Our identity is woven into every elegant solution we craft, every test we make pass, every problem we solve through pure logic and creativity. It's not just work, not just a craft – it's who we are." The emotional dimension of this shift should not be underestimated. Without acknowledging it, organisations risk replacing creative satisfaction with administrative oversight, leading to disengagement and erosion of craftsmanship.
Safe AI practice must therefore re-centre human responsibility while preserving the creative and technical aspects of development. Developers become custodians of quality, ethics, and security, preventing automation bias from cascading into production – while continuing to design, build, and innovate. This is not a reduction of their role but its expansion. As in editorial work, mastery now consists not only in producing more words or lines of code but in ensuring that every output – whether human-authored or AI-assisted – conveys the intended meaning securely and responsibly.
The curator's challenge
If the developer's role is shifting toward greater curatorial responsibility, the next question becomes how effectively individuals and organisations can perform that evaluative function. Early research suggests that this capability is not yet mature. The 2025 METR study exposed a recurring misalignment between perceived and actual productivity, showing that experienced developers "predicted 24% speed increase" when using AI tools but "in reality were 19% slower" (METR 2025). This perception gap matters because it shapes organisational decision-making. Teams that believe they are moving faster often overlook the growth of technical debt, security exposure, and review bottlenecks.
A similar cognitive distortion has been identified in behavioural studies of automation bias. When systems produce outputs that appear polished and coherent, developers often accept AI-generated code as correct without full review, a pattern consistent with automation bias, which describes the human tendency to overtrust machine output when it appears professional (CSET 2024)³. This bias is particularly acute in software engineering, where syntactic correctness can conceal semantic flaws. In a comparative evaluation of code generation models, including GPT-4 and Code Llama, CSET researchers found that "almost half of the code snippets produced by these five different models contain bugs that are often impactful and could potentially lead to malicious exploitation." Developers frequently accepted this code as secure, showing how presentation fluency can disguise substantive risk.
The emerging challenge of agentic AI
The curatorial challenge intensifies as AI tools evolve from passive code generators to active agents. Modern development assistants increasingly have access to code execution environments, browser capabilities, Model Context Protocol (MCP) integrations, and file system operations beyond the editor. This expansion fundamentally transforms the threat model.
Where traditional AI assistants could only suggest potentially malicious code that a developer might accept, agentic systems can execute that code directly if granted the necessary permissions. The attack surface broadens considerably: local execution risks, lateral movement, data exfiltration via browsing, supply chain compromise, etc.
Importantly, these risks exist even when the AI model itself has no malicious intent. The danger lies in prompt injection attacks (where carefully crafted inputs in documentation or code comments trick the agent into executing harmful operations), overly permissive configurations, inadequate logging, and developer assumptions that AI actions are as safe as AI suggestions.
The curator's challenge, therefore, involves resisting the machine's confidence and cultivating critical thinking.
In other words, effective AI use demands meta-cognitive skills such as the ability to question apparent efficiency and to test code not only for correctness but for integrity. It also requires establishing appropriate boundaries for what agents are permitted to do.
Safe AI use depends on upskilling developers to become curators, enabling them to combine domain expertise with disciplined scepticism. Without them, productivity gains will remain superficial.
Governance that enables the curator role
Responsible AI use in software development is not primarily a matter of restriction or compliance; it is about creating the organisational conditions that make careful evaluation the path of least resistance. Policies and governance frameworks must be designed to enable the developer-as-curator model rather than to constrain it.
European regulation already sees this shift. The EU AI Act (Regulation (EU) 2024/1689) requires organisations to document when and how AI is used in development and to ensure that human oversight remains demonstrable throughout the lifecycle of high-risk applications (European Parliament, 2024)⁴. These obligations implicitly define the developer's role as an accountable evaluator, namely, someone who understands the origin, logic, and implications of AI-generated output. The European Commission's Digital Strategy emphasises "AI systems that are safe, transparent, traceable, non-discriminatory and under human oversight," ensuring that all AI-assisted decision-making is accountable (European Commission 2024)⁵.
The General Data Protection Regulation (GDPR) reinforces the same principle in a different domain. Articles 5 and 25 stipulate that data processing must follow the principles of purpose limitation and data protection by design and by default (European Parliament and Council 2016)⁶. When developers use AI tools that interact with code containing customer or personal data, they must be able to explain, at the code level, where personal data might flow and how it is protected. Unmanaged AI interfaces that process such information without a valid Data Processing Agreement (DPA) violate these obligations. As the European Data Protection Board has clarified, responsibility for lawful and transparent data processing remains with the data controller, even when AI tools are employed. Compliance obligations cannot be delegated to the provider of the AI system; the organisation deploying it must ensure that data protection principles are implemented "throughout the lifecycle of the AI system" (European Data Protection Board 2024)⁷.
Practical governance, therefore, requires several enabling structures.
First, managed AI environments, such as GitHub Copilot Enterprise⁸, with organisational codebases and operate under approved DPAs. They allow the AI to generate within the context of established patterns and architectural conventions, reducing the likelihood of generic or insecure suggestions.
Second, prompt curation should become an explicit organisational practice. Tools like Cody⁹ enable teams to maintain internal libraries of effective prompts and their verified outputs, turning mere experimentation into organisational knowledge. This not only improves quality but also supports compliance by providing an auditable record of AI interaction.
Third, outcome-based metrics must replace output metrics. Safe AI use cannot be demonstrated through adoption rates or lines of code generated; it must be evaluated through indicators such as defect density, maintainability, and time-to-understanding for new contributors.
Fourth, agent capability controls become essential as AI assistants gain execution capabilities. Organisations should implement techniques like the principle of least privilege (restricted default configurations), sandboxing and isolation, action logging and auditability, human-in-the-loop for sensitive operations, network restrictions, to name a few.
Governance that supports the curator role recognises that human oversight is not a box-ticking exercise but a professional discipline. It requires developers to have a deeper understanding of security, privacy, and context than ever before.
The uncomfortable truth is that Europe's "restrictive" regulations might be the only thing preventing a wave of security breaches and GDPR violations from AI-generated code. The real question isn't whether regulation slows innovation but whether American companies will learn this lesson before or after their first major incident.
Supporting the curator role in practice
Safe AI use ultimately depends on how teams apply governance principles in their daily work. Policies define accountability, but culture determines whether responsible practice takes root. The Nordic work ethic provides a natural foundation for this transformation. Trust-based collaboration, collective responsibility, and sustainable work align well with the requirements of thoughtful AI integration (Nordic Council of Ministers 2023).
In this light, a practical starting point is rethinking how teams review code. In the context of AI-assisted development, code review begins not with syntax but with origin and intent. Each contribution should disclose whether AI assistance was used, specify the model or provider, and reference the prompt and context that generated the initial draft. For agentic AI tools, this should also include what actions the agent executed and what permissions it was granted. This documentation fulfils the traceability principle outlined in Article 12 of the EU AI Act, which requires providers to "allow for the automatic recording of events (logs) over the lifetime of the system"¹⁰. Capturing this information in pull request templates embeds transparency directly into everyday workflow.
An effective review also distinguishes functional correctness from risk evaluation. AI-generated code should be reviewed not only for logic and style but also for potential vulnerabilities, data exposure, and compliance risks. Information Commissioner's Office explains why human review is important in the AI lifecycle: "AI decisions involve meaningful human review and checks, where appropriate, to mitigate eroding of privacy through selection bias and attempts to spoof controls or circumvent privacy measures. Human reviewers have appropriate knowledge, experience, authority and independence to challenge decisions." (ICO 2025)¹¹. In practice, this means checking input validation, dependency integrity, and privacy boundaries in line with GDPR Article 25, which requires "data protection by design and by default".
Secure development practices are increasingly supported by automated tools for static and dependency analysis, such as Black Duck or Snyk, which identify vulnerabilities and license risks in open-source components before deployment. These complement secret-scanning and targeted testing approaches that verify integrity and compliance across the development lifecycle.
Documentation and reasoning remain central to the curator's role. Developers should record where they diverged from or accepted AI suggestions and articulate the reasoning behind those choices. This habit, often called "explain-back," helps reviewers see whether the developer truly understands the code or has accepted it blindly. Code is maintainable when the reasoning is clear; when justification ends with "the model suggested it," revision is needed. Good documentation also shortens "time-to-understanding," a stronger indicator of quality than the amount of code produced.
Testing practices should account for the deceptive fluency of AI-generated code. Unit tests remain essential, but complementary methods, such as property-based testing, fuzzing with untrusted inputs, and regression tests for previously fixed defects, are key to revealing subtle errors that apparent coherence can hide. Reviewers should also ensure that personal or sensitive data never appears in prompts or logs, and that any AI tool used operates under a valid Data Processing Agreement. This approach aligns with the GDPR's accountability principle and reduces the risk of data exposure, one of the most common issues in unmanaged AI use.
The way forward
This piece addresses a critical question in current research on whether AI coding assistants deliver measurable productivity gains. DORA found that AI adoption now improves software delivery throughput, and more than 80% of respondents report increased productivity (DORA, 2025, pp. 4, 30). However, these gains come with trade-offs. The same research shows AI still increases delivery instability (DORA 2025, p. 4). The productivity gains are real, but without proper organisational foundations, they can be lost to what DORA calls "downstream chaos" (DORA 2025, p. 3)¹². Not because the tools are bad, but because organisations lack the cultural and structural capacity to use them safely. Moreover, as AI tools evolve from passive generators to active agents, the risk profile expands from "might generate bad code" to "might execute malicious operations with insufficient oversight."
Safe AI use depends on how effectively developers embrace their evolving role as curators of AI-generated material while maintaining their creative and technical contributions to system design and implementation. Policies and governance frameworks can define accountability, but only practice can sustain it. In this context, the developer's task extends beyond code production to include evaluation, interpretation, and justification – all while continuing to design, reason about, and build systems that require deep technical expertise. The measure of expertise lies not in speed or volume, but in discernment, namely, in each and every developer's ability to decide what to trust, what to revise, and what to reject—as well as what to create from scratch and how to architect solutions that AI cannot yet autonomously produce.
This curatorial responsibility aligns closely with Nordic professional culture, where trust, collective ownership, and long-term thinking are central to quality work. When developers treat AI outputs as first drafts to be examined, verified, and refined—while maintaining the craftsmanship of their own creative work—they translate abstract principles of transparency and oversight into daily routines. Review documentation, prompt libraries, and shared testing artefacts become evidence of understanding as much as of control.
In this model, governance frameworks such as the EU AI Act and the GDPR formalise what good engineering already demands. Compliance is not an administrative burden but a natural outcome of professional judgment applied consistently and transparently.
The developer as curator embodies the essence of safe AI use. Rather than being displaced by automation, developers expand their role to one that values both creation and comprehension, both innovation and responsibility. This shift represents not a loss of craftsmanship but its natural evolution—maintaining the ability to design and build while adding the capacity to thoughtfully integrate AI tools. True expertise now encompasses writing code, evaluating machine-generated suggestions, establishing appropriate boundaries for agent capabilities, and ensuring that what is written—whether by hand or with AI assistance—endures, remains secure, and can be trusted.
Acknowledgement
The author wishes to thank Even Lysen for the thought-provoking discussions that inspired deeper reflection on AI beyond its hype.
Alina E. Negulescu, Strategic Advisor on Responsible AI at Twoday
References
¹ METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 2025. Source: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study
² Vella, A. (2025) The Software Engineering Identity Crisis. LeadDev Magazine, April 2025. Source: https://annievella.com/posts/the-software-engineering-identity-crisis/
³ CSET (2024) "Cybersecurity Risks of AI-Generated Code." CSET Report, November 2024. Source: https://cset.georgetown.edu/publication/cybersecurity-risks-of-ai-generated-code
⁴ European Parliament (2024) Regulation (EU) 2024/1689 of the European Parliament and of the Council on Artificial Intelligence (AI Act). Official Journal of the European Union, 13 June 2024. Source: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32024R1689
⁵ European Commission, "Artificial Intelligence," Digital Strategy portal, last updated 2024. Source: https://digital-strategy.ec.europa.eu/en/policies/european-approach-artificial-intelligence
⁶ European Parliament and Council (2016) Regulation (EU) 2016/679 – General Data Protection Regulation (GDPR). Official Journal of the European Union, L 119/1. Source: https://eur-lex.europa.eu/eli/reg/2016/679/oj
⁷ European Data Protection Board (2024) Opinion 28/2024 on the Interplay between the AI Act and the GDPR. Brussels: EDPB, December 2024. Source: https://edpb.europa.eu/system/files/2024-12/edpb_opinion_202428_ai-models_en.pdf
⁸ GitHub (2024) Copilot Trust Center. Source: https://resources.github.com/copilot-trust-center
⁹ Sourcegraph (2025) Cody for Teams – Prompt Libraries and Enterprise Compliance. Source: https://sourcegraph.com/cody
¹⁰ EU Artificial Intelligence Act, Article 12: Record-Keeping. Source: https://artificialintelligenceact.eu/article/12/
¹¹ Information Commissioner's Office (ICO), Human review, Source: https://ico.org.uk/for-organisations/advice-and-services/audits/data-protection-audit-framework/toolkits/artificial-intelligence/human-review/
¹² DORA (2025), State of AI-Assisted Software Development, Source: https://cloud.google.com/resources/content/2025-dora-ai-assisted-software-development-report