While AI use, starting with generative AI, is spreading company-wide, no shortage of companies have failed to grasp "who is managing which AI, and how." However, the risks associated with AI use are becoming an issue directly tied to management liability that "I didn't know" no longer excuses.
This article organizes the risks lurking in AI use into two usage patterns and explains what actual harm can result if countermeasures are neglected.
Thinking about risk in terms of two usage patterns
Corporate AI use can broadly be divided into the following two patterns. Different risks lurk in each.
Pattern 1: Using existing AI tools (SaaS, etc.)
This is where employees individually use general-purpose AI tools such as ChatGPT to improve their work efficiency. There are mainly two risks lurking here.
- Information leaks: Confidential information or personal data leaking externally due to careless prompt input by employees
- Shadow AI: The spread of a situation where frontline staff use AI tools on their own that the company is not aware of
This is a structure in which voluntary use aimed at improving frontline productivity ends up accumulating risk in places beyond the reach of the management department's awareness.
Pattern 2: Contracted or in-house developed AI products
This is where a company builds its own AI models or RAG systems tailored to specific internal operations or customer-facing services. Two risks lurk here as well.
- Vulnerability risk: Building vulnerabilities into the system due to insufficient security measures during development
- Operational risk: Unexpected incorrect responses (hallucinations) or incidents occurring after launch due to insufficient testing
This shows that separate management challenges exist not only for "users" but also for "builders."
Three real harms that occur if countermeasures are neglected
If these risks are left unaddressed, the actual harms a company could face include the following.
- Suspension of business by a partner citing "lack of governance audit"
- Complete disposal or injunction of a system due to data misconduct (copyright or IP infringement) in an in-house developed AI
- A business improvement order resulting from the external leak of customer data or confidential information (in violation of the Act on the Protection of Personal Information)
None of these can be resolved simply by halting AI use itself; they are impacts at a level that affects business continuity. Suspended business ties directly to revenue, a fully discarded system leads to a loss of development costs, and a business improvement order leads to a collapse of external trust.
Why "I didn't know" doesn't hold up
What makes the risk of AI use tricky is that, in many cases, it arises from good faith on the front lines and a desire to improve productivity. Employees are not creating risk out of malice; rather, an accumulation of judgment calls such as "I'll use it because it's convenient" or "I'll skip this step because I want to develop faster" ends up becoming risk for the organization as a whole.
That is exactly why, rather than relying on individual vigilance, organizations need to grasp their AI usage and put rules and structure in place. This is a challenge in the domain of organizational control (AI governance) that cannot be addressed by technical safety measures (AI Safety) alone.
What to do first
The first thing needed in responding to AI risk is to "make visible" your company's AI usage. Identify the AI tools used within the company and any AI products being developed or operated, and organize their purpose of use, the data handled, and the anticipated risks.
Next, establish criteria for deciding whether use is permitted, a person responsible for management, and an approval and review flow. If AI use spreads while these remain unclear, it can lead not only to information leaks and incorrect responses but also to management risks such as insufficient explanation to business partners or findings in an audit.
If it is difficult for your company to carry out the inventory and develop rules on its own, it is effective to make use of the process for obtaining AI Governance C Certification (AI Governance Core Certification). Through C Certification, you can move closer to a state where AI use is "visualized, managed, and explainable."
Elith provides consistent, hands-on support from gap analysis through to developing an AI inventory and internal rules, building a structure, and reviewing operations after certification.
As a certified consulting firm for C Certification, Elith supports companies in building AI governance structures and obtaining C Certification. If you are struggling with formulating rules and guidelines for using AI in your organization, or with building internal structures, please feel free to reach out to us.
We also welcome inquiries at the stage of "I don't know where to start" or "I want to get a clear picture of where we currently stand." Even if the specifics haven't been decided yet, we will work with you to figure out an approach suited to your current situation.