How the use of algorithms affects a person's sense of responsibility

Artificial intelligence has significantly accelerated the code writing process; however, engineering leaders are facing a more complex question: what happens after coding is complete. At a roundtable dedicated to operations in the AI era, leaders from sectors such as logistics, food technology, healthcare, telecommunications, fintech, e-commerce, and higher education exchanged experiences on how to scale using AI while maintaining reliability, security, and control.

The general conclusion of the participants was that speed is no longer the main issue; now, managing this process is the key aspect. Ipsita Basu moderated the roundtable, and among the participants were specialists such as Mandjunat Nakshatri (Engineering Head at Mojro), Shobhit Suman (CPO HungerBox), Pawan Kumar A T (CTO Digii/DigiiCampus), Pranjal Nadani (Co-founder and CTO of Cautio), Kenil Paul (VP of Technology and Solutions at 6D Technologies), Rimjim Agrawal (Co-founder and CTO of BrainSightAI), Dhruv Rastogi (Chief AI Officer at MediAssist), Chandrakesh Pandey (Senior VP of Technology at Niyo Solutions Inc.), Achal Sharma (CTO of TechnoSport), Sunil Kumar (Engineering Director at Meesho), and Anshul Sharma (Engineering Director at Scaler).

The event was organized by groundcover and Data Signals with support from YourStory. On the organizing side were Paul Trebe (Head of Sales and Partnerships at groundcover) and Satish Dange (Founder of Data Signals).

Participants agreed that AI has reduced build time, but the pressure has shifted to subsequent stages. When agents generate large volumes of code, questions arise about who will review, test, and control dozens of daily deployments. Many leaders noted that their teams are now spending more time on concept development, context, and management than on the actual component writing. The effectiveness of agents directly depends on the quality of the context they receive, and few organizations possess repository knowledge, documentation, and templates in a format that an agent could reliably use.

During the discussion, a consensus was reached regarding a progressive approach: reversible, read-only tasks, such as data retrieval or report generation, can be delegated to agents with minimal difficulty. However, irreversible actions, such as data deletion or database updates, must remain under human control. Assessing the scope of impact is also crucial; if a faulty call could affect end-users or production systems, a human must handle it.

One participant described a 'four-eyes' model where AI performs reading and analysis, deterministic rules provide clear 'yes' or 'no' answers, and the final approval is given by a human. Another speaker proposed a risk assessment system based on severity and frequency of occurrence, allowing human oversight to focus on medium and high-criticality elements. Some drew a line through specialization, allowing engineers to freely use agents in their domain while keeping production responsibility with humans capable of troubleshooting at any time.

A more optimistic picture also emerged: instead of waking up an engineer, an agent can investigate an incident and provide a report by morning, leaving the decision to the human. Participants emphasized that trust will build gradually, similar to how it happened with online payments, and this boundary will constantly shift.

The issue of accountability received firm agreement: this duty cannot be delegated to a model. Leaders remain responsible for the actions of agents. One participant noted that AI acts as an intelligence multiplier, meaning errors amplify as quickly as good decisions.

The discussion on skill obsolescence sparked more lively debates. Some saw it as simply a shift in required competencies, analogous to how arithmetic changed with the advent of calculators. Others, particularly in healthcare, feared the loss of expertise needed to evaluate the results of AI work. One practical solution was the conscious definition of areas where AI handles drafts and where humans perform the work.

For leaders in the telecom and medical fields, data control is determined at the design stage. Models cannot simply operate in a public cloud if regulatory norms, requirements for tenancy separation, and residency rules apply, as one mistake can lead to serious fines. Others emphasized governance from the outset, covering data intake, what is used for training versus output, and methods for data masking or de-identification. Concerns about intellectual property and bias also led to a call to evaluate AI tools as thoroughly as new employees.

Observability permeated all discussed topics. Teams spoke about the need to track inputs, decisions, and task handoffs by agents to understand the cause of their actions and correct them. One e-commerce team learned the cost of insufficient investment: a rating change left all dashboards green until the business impact manifested days later, and root cause analysis took over a week. Since then, they created a change log, lineage system, system graph, and a debugging agent that suggests ranked hypotheses.

From an observability perspective, it can no longer exist separately from development. Paul Trebe stated plainly: 'You can no longer separate observability from development.' He added that agents require context-rich data to act correctly, and the agents themselves must be monitored, as AI traces can reach 500,000 lines, which realistically only other agents can parse.

No one presented a ready-made guide. Questions remain open regarding methods for measuring agent reliability and ROI, data and intellectual property protection, and maintaining sufficient sharpness in engineers to challenge AI outputs. Nevertheless, the conversation pointed to a general direction: enabling developers to move fast, but first building protective barriers. Human control must be maintained over irreversible and high-risk decisions, and automation should expand as trust is earned. Many participants argued that inaction is unacceptable, as competitors will not wait. The true test of the AI era is not how fast teams can build, but how confidently they can operate what they have built.

In recent years, improving AI performance has depended on the ability to write higher-quality prompts. However, this paradigm is rapidly changing as AI systems begin to reason, make decisions, and perform actions within entire workflows autonomously.

At the TechSparks 2026 event, YourStory's flagship event, Dr. Vignesh Subrahmaniam, an AI lead scientist at Intuit, held a discussion with Aditya Charlie, Editor-in-Chief of YourStory and The CapTable. The discussion, titled 'Beyond the Prompt: Building AI That Thinks, Acts, and Learns,' focused on what developers need to rethink as intelligence becomes increasingly autonomous.

The session began with questions about Subrahmaniam's typical workday and a request to simplify fintech and AI concepts for the audience. He used a chess analogy, asking: if an algorithm is not taught the rules of chess but shown a million completed games with winners, can it deduce both the rules and the most probable winning strategies?

Intuit's team applies a similar approach to support small businesses, as the company serves both consumers and accountants. Subrahmaniam noted: 'We study the game of business by analyzing data, real actions, and the results that enterprises undertake.' The goal is to understand the factors that contribute to business success in customer acquisition, rapid growth, and improved cash flow.

The conversation then shifted to the paradigm shift for developers: moving from years of studying how to write smarter prompts to designing agents given a goal, which they determine the steps to achieve themselves. Subrahmaniam compared prompt engineering to the long history of programming languages, starting from binary code and assembly up to Fortran, C, and Python. Large language models have allowed anyone to program in simple English, but this method has its limitations.

Researchers quickly realized that intelligence should not reside only within the model. The model must act as an orchestrator, breaking down a complex task into a sequence of actions and tool calls. Real intelligence moves into the called tools; and these tools must also be designed differently: 'not so that a human, or a software developer, or anyone else can use them, but in such a way that the LLM orchestrator can actually use them.'

This requires creating highly precise tools with extremely deterministic outcomes. For developers, mastery shifts from clever phrasing to reliable components that the model can call repeatedly without surprises.

Regarding sources of new data, Subrahmaniam openly stated that training on text data is largely solved. The more complex task is training on multimodal sources and real-world actions. He cited the example of how a young child primarily learns through what they see, hear, and observe, rather than reading, asserting that incorporating such real-world signals into AI systems will help models move beyond text generation and start suggesting actions. He called this the next frontier.

When considering regulated industries, he took the audience back 18 years, to one of the first machine learning labs in Bangalore where he worked on logistic regression and multilayer perceptrons. Back then, the biggest problem was not the algorithm itself. He recalled: 'Actually, it was the quality of the data.'

This problem has not changed. Whether a team is training simple regression or fine-tuning a transformer, the principle of 'garbage in, garbage out' remains valid. According to him, companies capable of collecting, aggregating, and storing the highest quality data will gain an advantage.

During a short Q&A session, one participant asked how restrictions on storing financial data affect Intuit's operations. Subrahmaniam named consent as the foundation of any training system and referenced the Framingham study, where residents of Massachusetts were allowed to track their health over time, which shaped the modern understanding of cardiovascular disease. He added that companies are also obligated to explain the benefits of data usage to the general public.

As the conversation concluded, the overarching idea was that as AI transitions from reacting to acting, foundations become more important than interfaces. Prompts will remain a starting point, but the real work now lies in creating robust tools that agents can call, supplying systems with high-quality data, and obtaining the consent that allows them to learn. It is here, beyond the prompt, that the next wave of AI will be won for developers.

Despite the ubiquity of automation, it does not guarantee that it benefits customers. A chatbot unable to answer a question, an interactive voice response (IVR) menu that doesn't match the issue, or an automated response leading to a dead end can turn a simple query into a highly unpleasant experience. Even if the interaction is technically logged as successfully completed, this is far from the case for the customer.

According to the CallMiner CX Landscape Report 2026, there is a significant gap. While 99% of organizations use automation in customer interactions, only 23% claim that these processes are both highly automated and continuously optimized based on Customer Experience (CX) data. Furthermore, only 24% describe the resulting customer experience as genuinely positive. Thus, the focus has shifted from implementing automation to ensuring it meets current customer needs.

What defines CX automation technologies?

Despite the temptation to view this gap as purely a technological problem solvable by replacing a platform or using a smarter model, organizations closing this gap do not necessarily employ the newest automation tools. They continuously refine automation by using feedback and data gathered from real conversations, rather than relying on assumptions made during initial workflow design.

Without such guidance, most teams resort to one of three reactive habits. Some continue tuning the system without knowing what is failing. Others completely abandon the platform and start over, hoping new technology will solve a problem that was never diagnosed. The third group reverts to purely human-based support, dismantling years of CX investment. All three approaches miss the same critical step: understanding the core issue within the conversation itself.

How does customer analytics change the approach to work?

Customer analytics provides a holistic view of what is happening across all customer interactions, transforming automation from a tactic into a strategy. It allows visibility into where automation truly helps and where the organization merely assumes its usefulness. This tool identifies behavioral and contextual signals, such as repeat contact on the same topic or a shift in mood mid-dialogue, which a static customer profile could not capture. Moreover, it links technical metrics, like retention and deflection rates, with outcome metrics, such as Customer Satisfaction (CSAT) and first-contact resolution, preventing a situation where a well-looking dashboard hides an unhappy customer.

This distinction is more significant than often acknowledged. A high retention rate does not guarantee a positive outcome. A bot might technically complete an interaction even if the customer leaves without a resolution. Conversation analytics reveals this gap by showing what occurred during the interaction, rather than just stating whether it ended without human involvement. This applies to companies refining long-used automation as well as teams deploying their first virtual agent. The key question is not, 'Have we automated enough?', but rather, 'Is this automation based on what customers are saying today, or what they said when the workflow was created?'

Measuring automation effectiveness beyond mere efficiency. This does not mean efficiency ceases to matter. It means that the strongest automation strategies reduce unnecessary effort for both customers and employees while protecting moments where human judgment and empathy create the most value. Adhering to this balance correctly distinguishes automation that customers tolerate from automation that they trust.

A new technical document from the company explains how to use conversation analytics to determine where automation should be applied, to design experiences around actual customer behavior, and to measure success based on customer outcomes, not just technical metrics. The full document is available, demonstrating how one organization implemented these principles, reducing call handling time by 78% while doubling capacity without increasing staff.