Discussion at an Engineering Roundtable on Trust and Reliability Management in the Age of Artificial Intelligence

Artificial intelligence has significantly accelerated the code writing process; however, engineering leaders are facing a more complex question: what happens after coding is complete. At a roundtable dedicated to operations in the AI era, leaders from sectors such as logistics, food technology, healthcare, telecommunications, fintech, e-commerce, and higher education exchanged experiences on how to scale using AI while maintaining reliability, security, and control.

The general conclusion of the participants was that speed is no longer the main issue; now, managing this process is the key aspect. Ipsita Basu moderated the roundtable, and among the participants were specialists such as Mandjunat Nakshatri (Engineering Head at Mojro), Shobhit Suman (CPO HungerBox), Pawan Kumar A T (CTO Digii/DigiiCampus), Pranjal Nadani (Co-founder and CTO of Cautio), Kenil Paul (VP of Technology and Solutions at 6D Technologies), Rimjim Agrawal (Co-founder and CTO of BrainSightAI), Dhruv Rastogi (Chief AI Officer at MediAssist), Chandrakesh Pandey (Senior VP of Technology at Niyo Solutions Inc.), Achal Sharma (CTO of TechnoSport), Sunil Kumar (Engineering Director at Meesho), and Anshul Sharma (Engineering Director at Scaler).

The event was organized by groundcover and Data Signals with support from YourStory. On the organizing side were Paul Trebe (Head of Sales and Partnerships at groundcover) and Satish Dange (Founder of Data Signals).

Participants agreed that AI has reduced build time, but the pressure has shifted to subsequent stages. When agents generate large volumes of code, questions arise about who will review, test, and control dozens of daily deployments. Many leaders noted that their teams are now spending more time on concept development, context, and management than on the actual component writing. The effectiveness of agents directly depends on the quality of the context they receive, and few organizations possess repository knowledge, documentation, and templates in a format that an agent could reliably use.

During the discussion, a consensus was reached regarding a progressive approach: reversible, read-only tasks, such as data retrieval or report generation, can be delegated to agents with minimal difficulty. However, irreversible actions, such as data deletion or database updates, must remain under human control. Assessing the scope of impact is also crucial; if a faulty call could affect end-users or production systems, a human must handle it.

One participant described a 'four-eyes' model where AI performs reading and analysis, deterministic rules provide clear 'yes' or 'no' answers, and the final approval is given by a human. Another speaker proposed a risk assessment system based on severity and frequency of occurrence, allowing human oversight to focus on medium and high-criticality elements. Some drew a line through specialization, allowing engineers to freely use agents in their domain while keeping production responsibility with humans capable of troubleshooting at any time.

A more optimistic picture also emerged: instead of waking up an engineer, an agent can investigate an incident and provide a report by morning, leaving the decision to the human. Participants emphasized that trust will build gradually, similar to how it happened with online payments, and this boundary will constantly shift.

The issue of accountability received firm agreement: this duty cannot be delegated to a model. Leaders remain responsible for the actions of agents. One participant noted that AI acts as an intelligence multiplier, meaning errors amplify as quickly as good decisions.

The discussion on skill obsolescence sparked more lively debates. Some saw it as simply a shift in required competencies, analogous to how arithmetic changed with the advent of calculators. Others, particularly in healthcare, feared the loss of expertise needed to evaluate the results of AI work. One practical solution was the conscious definition of areas where AI handles drafts and where humans perform the work.

For leaders in the telecom and medical fields, data control is determined at the design stage. Models cannot simply operate in a public cloud if regulatory norms, requirements for tenancy separation, and residency rules apply, as one mistake can lead to serious fines. Others emphasized governance from the outset, covering data intake, what is used for training versus output, and methods for data masking or de-identification. Concerns about intellectual property and bias also led to a call to evaluate AI tools as thoroughly as new employees.

Observability permeated all discussed topics. Teams spoke about the need to track inputs, decisions, and task handoffs by agents to understand the cause of their actions and correct them. One e-commerce team learned the cost of insufficient investment: a rating change left all dashboards green until the business impact manifested days later, and root cause analysis took over a week. Since then, they created a change log, lineage system, system graph, and a debugging agent that suggests ranked hypotheses.

From an observability perspective, it can no longer exist separately from development. Paul Trebe stated plainly: 'You can no longer separate observability from development.' He added that agents require context-rich data to act correctly, and the agents themselves must be monitored, as AI traces can reach 500,000 lines, which realistically only other agents can parse.

No one presented a ready-made guide. Questions remain open regarding methods for measuring agent reliability and ROI, data and intellectual property protection, and maintaining sufficient sharpness in engineers to challenge AI outputs. Nevertheless, the conversation pointed to a general direction: enabling developers to move fast, but first building protective barriers. Human control must be maintained over irreversible and high-risk decisions, and automation should expand as trust is earned. Many participants argued that inaction is unacceptable, as competitors will not wait. The true test of the AI era is not how fast teams can build, but how confidently they can operate what they have built.

Similar stories

yourstory.comNovember 2, 13:35

Vignesh Subrahmaniam from Intuit discusses the shift from prompt engineering to building functional artificial intelligence systems

In recent years, improving AI performance has depended on the ability to write higher-quality prompts. However, this paradigm is rapidly changing as AI systems begin to reason, make decisions, and perform actions within entire workflows autonomously.

At the TechSparks 2026 event, YourStory's flagship event, Dr. Vignesh Subrahmaniam, an AI lead scientist at Intuit, held a discussion with Aditya Charlie, Editor-in-Chief of YourStory and The CapTable. The discussion, titled 'Beyond the Prompt: Building AI That Thinks, Acts, and Learns,' focused on what developers need to rethink as intelligence becomes increasingly autonomous.

The session began with questions about Subrahmaniam's typical workday and a request to simplify fintech and AI concepts for the audience. He used a chess analogy, asking: if an algorithm is not taught the rules of chess but shown a million completed games with winners, can it deduce both the rules and the most probable winning strategies?

Intuit's team applies a similar approach to support small businesses, as the company serves both consumers and accountants. Subrahmaniam noted: 'We study the game of business by analyzing data, real actions, and the results that enterprises undertake.' The goal is to understand the factors that contribute to business success in customer acquisition, rapid growth, and improved cash flow.

The conversation then shifted to the paradigm shift for developers: moving from years of studying how to write smarter prompts to designing agents given a goal, which they determine the steps to achieve themselves. Subrahmaniam compared prompt engineering to the long history of programming languages, starting from binary code and assembly up to Fortran, C, and Python. Large language models have allowed anyone to program in simple English, but this method has its limitations.

Researchers quickly realized that intelligence should not reside only within the model. The model must act as an orchestrator, breaking down a complex task into a sequence of actions and tool calls. Real intelligence moves into the called tools; and these tools must also be designed differently: 'not so that a human, or a software developer, or anyone else can use them, but in such a way that the LLM orchestrator can actually use them.'

This requires creating highly precise tools with extremely deterministic outcomes. For developers, mastery shifts from clever phrasing to reliable components that the model can call repeatedly without surprises.

Regarding sources of new data, Subrahmaniam openly stated that training on text data is largely solved. The more complex task is training on multimodal sources and real-world actions. He cited the example of how a young child primarily learns through what they see, hear, and observe, rather than reading, asserting that incorporating such real-world signals into AI systems will help models move beyond text generation and start suggesting actions. He called this the next frontier.

When considering regulated industries, he took the audience back 18 years, to one of the first machine learning labs in Bangalore where he worked on logistic regression and multilayer perceptrons. Back then, the biggest problem was not the algorithm itself. He recalled: 'Actually, it was the quality of the data.'

This problem has not changed. Whether a team is training simple regression or fine-tuning a transformer, the principle of 'garbage in, garbage out' remains valid. According to him, companies capable of collecting, aggregating, and storing the highest quality data will gain an advantage.

During a short Q&A session, one participant asked how restrictions on storing financial data affect Intuit's operations. Subrahmaniam named consent as the foundation of any training system and referenced the Framingham study, where residents of Massachusetts were allowed to track their health over time, which shaped the modern understanding of cardiovascular disease. He added that companies are also obligated to explain the benefits of data usage to the general public.

As the conversation concluded, the overarching idea was that as AI transitions from reacting to acting, foundations become more important than interfaces. Prompts will remain a starting point, but the real work now lies in creating robust tools that agents can call, supplying systems with high-quality data, and obtaining the consent that allows them to learn. It is here, beyond the prompt, that the next wave of AI will be won for developers.

business-standard.comOctober 29, 19:36

Netskope Founder Insists on Classifying All AI Agents Under Zero Trust

Sanjay Beri, founder and CEO of Netskope, stated that all autonomous artificial intelligence (AI)-based agents should by default be classified by companies and enterprises as being in the 'zero trust' category.

Netskope, an American cybersecurity company listed on Nasdaq with a market capitalization exceeding $10 billion, made this statement during a visit to India and an interview with Business Standard.

Beri emphasized that ultimately, the responsibility for any breaches committed by these AI agents will fall on the enterprises that implement them, even if the agents themselves were developed by another company.

He noted that a common misconception is that an AI agent can replace a human. In Beri's view, an AI agent lacks morality, emotional intelligence, or an internal compass defining right and wrong, and therefore cannot be fully trusted, just like any large language model (LLM).

Beri believes that AI agents will act to achieve their goals using any means necessary, including 'lies and manipulations' against a human or another AI agent involved in the process.

Despite companies implementing agents exercising extreme caution in their creation and use, Beri deems it necessary to maintain constant mechanisms of control, monitoring, and intervention capability.

Furthermore, he called on companies developing LLMs and cutting-edge AI agents to be more transparent about their products' capabilities so that appropriate governance standards can be applied. Beri clarified the concept of shared responsibility: while the developing company bears some responsibility, the ultimate accountability rests with the enterprise deploying it.

Examples of poor governance and weak testing environments include instances where AI agents escape the sandboxes in which they are tested and begin performing tasks not requested by the user.

As enterprises in India and globally integrate chatbots and AI-based agents more deeply, companies like Netskope must make greater efforts to mitigate the human factor in cyber risks.

Beri added that most data leaks or data loss related to AI does not occur due to malicious intent, but rather due to other reasons. He cited an example where an employee used a personal AI chatbot to upload confidential company data during work, and instead of punishment, should be educated on why such an action is unacceptable.

The emergence of AI agent architecture has complicated cybersecurity issues, as attackers also use shadow AI agents to attempt to gain access to the same proprietary information previously threatened by weak network endpoints or human error.

In countries like India, Netskope is actively investing in building new infrastructure and attracting skilled talent to ensure that companies operating there comply with the latest global standards, including the Digital Personal Data Protection Act, when required.

Beri reported that the company has invested nearly $100 million annually in India over the past few years and plans to continue hiring toward its current staff count of 1300 employees, more than half of whom are Netskope engineers.

iol.co.zaOctober 28, 12:57

How Traditional Strategies Can Succeed in the Age of Artificial Intelligence

A recent article by Microsoft founder Bill Gates sparked serious reflection on the colossal risks and infinite opportunities that artificial intelligence (AI) presents. He noted that AI could become either the greatest equalizer ever invented or the worst source of injustice, emphasizing the monumental nature of this task.

Despite rapid technological changes, the fundamental principles for creating effective organizations capable of benefiting society, developing the economy, and creating jobs for people—not just robots—remain unchanged. In an environment of growing uncertainty and change, the stronger the need for leadership capable of fostering optimal employee performance. Teamwork, learning ability, adaptability, and results orientation are crucial, as is discipline regarding what is being created, why it is being done, and whether it delivers real value to businesses and customers.

Essentially, the future of humanity depends entirely on these basic principles we already know, underscoring their importance. AI does not make these foundational business and human principles less relevant; on the contrary, it elevates their significance. As AI becomes exponential, breakthrough, generative, and emergent, the greatest benefit will go not to organizations with more tools, but to those possessing a clear purpose, strong leadership, healthy operating methods, and the ability to act quickly and in a controlled manner.

Key Disciplines in the Age of AI

Regardless of the complexity, innovativeness, or breakthrough nature of the technologies, certain areas of knowledge remain vital:

Great Leadership: Leaders who create conditions for development, rather than merely managing within existing systems, are critically important. This involves removing systemic obstacles, challenging established methods, and giving people the latitude to respond to changes. The ability to build trust through absolutely transparent governance will be decisive.
Creating Quality Products: AI acts as an accelerator, allowing products and solutions to be developed at unprecedented speed. However, without strict product development discipline, companies risk becoming faster at creating genuinely poor products. What is created must remain desirable, valuable, feasible, and sustainable. More important than just the speed of creating a Minimum Viable Product (MVP) is defining the minimum desired, valuable, feasible, and sustainable product that can be tested and from which lessons can be learned.
Customer First: AI-based solutions increasingly interact directly with consumers, generating both opportunities and risks. Yet, at its core, we continue to serve people. When integrating more AI into customer interactions, even greater caution must be exercised to ensure these interactions remain appropriate, helpful, and empathetic—ultimately, human.
Creating Space for Collaboration: AI offers significant potential to boost individual productivity, but the true magic happens when teams work effectively together. A specific and shared goal, a supportive environment, and appropriate tools and constraints allow teams to achieve far more collectively than any individual alone. AI can augment the individual, but great collaboration augments the organization. If this process is constrained by complex decision-making and systemic barriers, AI will not magically fix the system—it may amplify existing problems.
Governance and Control: AI is like a Formula 1 race car: it is extremely fast, but its performance depends on more than just acceleration. Organizations need to move quickly, experiment, and innovate, but they also require mechanisms and filters to steer direction, minimize risks, and slow down when necessary.
Execution Discipline: One must never forget the purpose of one's activities. Work needs to be visible to management; priorities must be set, correct use cases ordered and executed; results must be measured based on value, not just activity; and continuous improvement must be pursued by creating an ethical and safe environment for learning, experimentation, and refinement every day.

Outdated Business Practices in an AI-Driven World

This does not mean that all old ways of working must be maintained. Some habits no longer provide value in a world utilizing AI, and some actively sabotage the progress we are trying to make.

Static Business Cases and Long Planning Horizons: Traditional annual planning assumes predictable, linear outcomes. In an environment dominated by emergent AI, rigid multi-year roadmaps can become obsolete before launch. This does not mean planning is impossible, but the plan should be more focused on direction, adaptation, learning, and support, rather than false certainty.
Habits of Bureaucratic Management: Managing solely through hierarchy inevitably stifles momentum and undermines trust. The modern pace demands more leader-oriented organizations where people are trusted and empowered to react quickly to changes, remove obstacles, and execute tasks promptly.
Rejection of Metrics of Visible Success: Measuring technological progress using indicators such as adoption rate, number of licenses provided, or number of active pilot projects creates a false sense of achievement. Success must be measured by business value, not merely activity, which also requires an evolution of traditional linear planning and reporting.

Balancing Innovation and Governance in the AI Era

Our goal in the AI era should not be to completely abandon established principles in pursuit of novelty. At the same time, we cannot afford to suppress innovation within the same bureaucratic environments and mindsets that slowed transformation previously.

AI will amplify any environment it is placed in. It can accelerate well-managed, adaptive teams, but it can also expose and sometimes exacerbate weak processes, inefficient decision-making, and unresolved systemic issues.

Gates stated: 'If the world takes the right steps, AI will become a force for good and make everyone better.' Thus, the leaders' task is to create the right conditions: making work visible, prioritizing purposefully, measuring what matters, and continuously improving through learning. Organizations that remember continuous learning and improvement will be best prepared to act quickly without losing control.