Four months after the European Parliament adopted the AI Act, the European Union Agency for Cybersecurity (ENISA) has released a report warning about the risks associated with generative AI. As AI continues to play an increasingly prominent role in our daily lives, the risks associated with this emerging technology are not always well understood.

An ever-increasing role

Artificial intelligence is playing an increasingly significant role in our daily lives. Chat GPT, which is capable of generating text based on the questions asked to it, is an excellent example of this.

This conversational AI developed by OpenAI has become the fastest-growing consumer application in history. According to UBS, Chat GPT had 100 million active users just two months after its launch, a milestone that even the viral social network TikTok took 9 months to achieve.

However, despite its popularity, users are not always aware of the risks it poses. In March 2023, Chat GPT was taken offline for several hours due to a serious data breach involving sensitive user information.

The European Union is legislating

Faced with the widespread phenomenon of generative AI, such as Chat GPT or its Google counterpart, Bard, the European Parliament adopted the AI Act on Wednesday, June 14, 2023. This is a regulatory text for AI aimed at creating a regulatory framework for market deployment with a focus on issues of security, health, and fundamental rights.

In its report dated October 19, 2023, the European Union Agency for Cybersecurity (ENISA) also identified and warned against several typologies of risks associated with AI. It particularly highlights the increase in cyberattacks.

More sophisticated attacks

Indeed, with AI, cyberattacks can be more effective as they become more realistic and operate on a larger scale.

The era of smishing attempts (phishing via SMS) or phishing emails filled with spelling mistakes and inconsistencies is a thing of the past. With generative AI, cybercriminals can prepare more convincing scams. The same applies to phone scams: with the development of deepfakes, they can impersonate other individuals by cloning their voices!

Some risks to data

The risks associated with generative AI also lie in the information entrusted to these tools. It is essential to be vigilant about what is requested from them, as they do not guarantee the confidentiality of this data.

Samsung recently experienced this firsthand when employees allegedly entered confidential and strategic information into Chat GPT. As with any generative AI, this information was then transformed into training data for the AI to improve the accuracy of its future responses. This information could then be shared with other users outside the company to answer their questions.

Corporate secrets just a click away, and one can easily imagine the devastating impacts if they were to fall into the hands of a malicious individual.

The post Generative Artificial Intelligence: What Risks? appeared first on Erium.

How does the emergence of Artificial Intelligence impact confidentiality?

An essential principle of Artificial Intelligence technologies is algorithm learning. For an artificial intelligence algorithm to fulfill its function, it first needs to learn what it should do.

Following a learning curve principle, the algorithm multiplies and optimizes the pathways that enable it to perform its functions.

This learning phase requires massive volumes of data that serve as its training ground. The sources of information are diverse and rely on the daily use of its services, which promotes a continuous improvement of the calculation mechanisms. By injecting data into the system, users contribute to the learning databases of the algorithms.

From a cybersecurity perspective, this is where the risk lies. The offered services are extremely user-friendly, and anyone can use them to input anything.

At the beginning of 2023, Cyberhaven conducted a study on the use of ChatGPT by employees of companies. Although alarming, the result is not surprising: 2.3% of employees have already copied confidential information into the AI software developed by OpenAI.

This reality raises questions about the usage rules of the services offered, especially in their free versions.

Looking back at 4 very popular solutions:

ChatGPT – Text-generating AI developed by OpenAI (United States)

You can apparently ask it anything: from drafting a document to translating a text into any language. To use the service, you just need to log in and type your question, and the algorithm responds immediately.

Does the service use your information?

According to the information provided in the ‘terms of use,’ there are two scenarios. If the data comes from the API provided by OpenAI, then no, it does not use your information. However, if the data does not come from the API, then yes, it may use your information.

The question is as follows: which services are non-API and which ones use the API? The answer can be found on OpenAI’s blog.

In conclusion, if you use the ChatGPT service, the well-known one, the default policy allows the editor to reserve the right to use the data you input into it.

However, there is a form provided by OpenAI to request that your data injected into the ChatGPT service not be utilized.

MidJourney – Image generation AI on Discord (United States)

MidJourney is an incredible algorithm. You provide it with textual instructions that it translates into images with impressive finesse and creativity. The operating principle is relatively simple. After creating an account, you log in to Discord to give artistic creation orders (referred to as /imagine prompt) in the form of keywords. You can also input a photo that belongs to you as input. Many users, in fact, input their own photos, including ID photos, family pictures, and party photos, to be modified by the algorithm.

Does the service use your information?

Yes, without any restrictions, and this is explicitly mentioned in the article ‘Rights you give to MidJourney’ in the terms of use of the service.

So, there is no ambiguity; if you submit information to MidJourney, the data becomes the property of MidJourney.

DeepL – Language-to-Language Translation AI (Germany)

A highly popular service, with its extremely user-friendly interface, DeepL is used by thousands of people every day to translate texts into various languages.

The usage is straightforward: you simply input a text and select the desired output language.

Does the service use your information?

No and yes! The service does not retain the memory of the input texts, only if you use it in a paid Pro version. The free service, on the other hand, retains and reserves the right to utilize the injected data.

ElevenLabs – Voice Generation AI (United States)

ElevenLabs offers a voice generation service. By the end of March 2023, the service was made available to the public in a Beta version with a very simple interface.

Does the service use your information?

In its free version, yes, the injected data is retained and utilized by the service.

To conclude

Is AI a revolution, and will it become a permanent fixture in the digital landscape? Absolutely yes, it’s already the case, and it will continue to accelerate by offering more and more AI services to facilitate digital usage for both the general public and organizations. For instance, in March 2023, Microsoft announced the integration of OpenAI’s algorithms into its Azure environment and the integration of Copilot (another OpenAI solution) into the Microsoft Office 365 suite. This is an enduring trend.

However, the emergence of these new uses also comes with risks, particularly for data privacy. It is essential to remind users of best practices and raise awareness of the use of these tools:

1. Do not upload a confidential document to be translated by an algorithm without knowing what happens to the source afterward.
2. Do not upload photos of third parties into a tool without their consent.
3. Do not ask an algorithm to process personal data without verifying their compatibility with GDPR and security policies.

Beyond its undeniable contribution to data processing, artificial intelligence also provides internet users with the opportunity to exercise common sense and discernment.

The post AI and Privacy Denial appeared first on Erium.

BAS highlights the vulnerabilities that Red Teams exploit

The two approaches have different end goals.

The Red Team follows a path of compromise known as the “path of least resistance”: spending as little time as possible on the targeted infrastructure, leaving minimal traces, taking the shortest route, limiting movements, and so on.

By achieving its objective, it highlights very targeted and localized vulnerabilities that will subsequently be addressed. The BAS solution explores all conceivable paths, generating various levels of noise with the primary objective of verifying that what should be detected is indeed detected and effectively addressed by cyber teams.

In the process, the simulations produced by BAS highlight numerous failures, sometimes isolated, often overlooked, and easy to rectify. Here are a few examples:

Regarding detection:

• Failures or gaps in log collection and forwarding services (missing or faulty Syslog)
• Limitations in log strategy (log type, retention period, etc.)
• Misconfigurations in SIEM correlation scenario settings (inappropriate or inactive rules)
• Failure to detect a specific action by an EDR that should have detected it
• Failure or absence of alerts from detection consoles (SIEM, EDR, etc.)
• Obsolescence of CTI data (reactivation of a known C&C)
• Limitations related to alert thresholds for specific events (at what point, in terms of packets, ports, or targeted IPs, is a scan considered abnormal?)
• Presence of unnecessary and/or vulnerable open services
• Failure to detect data leakage by a Data Leak Detection service

Regarding response:

• Application errors in severity qualification event processes
• Delays in exporting or retrieving logs required for an investigation
• Limitations on access provided to analysts for their investigations
• Coordination of alert sharing between internal teams and the MSSP (Managed Security Service Provider)
• Coordination between security teams and IT teams to neutralize or contain the threat (network isolation of the source, disconnecting a malicious device, etc.)

BAS throws a wrench into the Red Team’s work

One of the benefits of Breach and Attack Simulation solutions is the training of defense teams (Blue Team). The more they face increasingly complex simulations, the more detection mechanisms will be optimized, the more effective the defenders’ processes and reflexes will become, and the faster and more relevant their investigations will be.

In summary, BAS will not replace Red Teams but will drive them to be better.

BAS = Panoramic Photo and Red Team = Photo with 24x Zoom

A well-executed Red Team is an undetected Red Team. And this result is what is reported: “if a highly skilled, well-organized attacker takes their time to target you, they will eventually succeed.”

This is a strong message, but it is becoming less audible to CEOs and auditors. Repeated for years, this message is well understood and absorbed. The proof: cybersecurity budgets increase every year, and technological components continue to stack up.

The question now being asked is about the real and overall effectiveness of these measures and investments. Since the Red Team focuses on a specific and well-defined path, it cannot provide this comprehensive picture. BAS and its more “massive” approach offer a response to this new question.

To measure, you need a precise rule

The concepts of measurement and ranking are increasingly prominent in assessing cybersecurity performance within organizations.

To measure the evolution of cybersecurity maturity, you need a baseline reference point that serves as a standard for comparison. In the realm of IT, comparison relies on precise and replicable technical criteria and actions. It’s crucial to compare apples to apples.

Consider a real-life scenario:

During an attack simulation, the SOC doesn’t detect the 3 persistence tactics automatically executed by the BAS. Upon analysis, it becomes apparent that the detection scenario is not correctly configured. This can happen, and the MSSP corrects it. A second test a few weeks later, the BAS replays the exact same technical event (flow, source, target, time, sequencing, etc.). This time, the scenario triggers an alert. Bingo! The scenario is then automatically retested periodically. This is done to verify that there is no regression, for continuous monitoring and assurance of ongoing compliance.

To conclude

If there’s one piece of advice to take away: if you’re considering mobilizing a Red Team, do it in environments where BAS solutions indicate that you excel.

It is in these specific contexts and environments that the human touch of a Red Team will bring you the most interesting results.

In this regard, BAS and Red Teams are unquestionably complementary and will remain so for a long time.

The post BAS versus Red Team : how attack simulation alters the landscape appeared first on Erium.