Homegrown Intelligence: AI Features for On-Prem MySQL Enterprise

Inside MySQL: Sakila Speaks

leFred and Scott sit down with Gaurav Chadha to explore MySQL AI, a new solution that brings advanced AI features available in HeatWave to organizations running MySQL Enterprise Edition on-premises. Discover how MySQL AI bridges the gap between cloud innovation and on-premise infrastructure, making transformative AI capabilities more accessible, secure, and efficient for teams that rely on MySQL Enterprise Edition wherever their databases reside.

--------------------------------------------------------------

Episode Transcript:

00:00.000 --> 00:25.000 Welcome to Inside MySQL: Sakila Speaks, a podcast dedicated to all things MySQL. We bring you the latest news from the MySQL team, MySQL product updates and insightful interviews with members of the MySQL community.

00:25.000 --> 00:32.000 Sit back and enjoy as your hosts bring you the latest updates on your favorite open source database. Let's get started.

00:32.000 --> 00:37.000 Welcome back to another episode of Inside MySQL: Sakila Speaks. Hi, I'm LeFred.

00:37.000 --> 00:38.000 And I'm Scott Stroz.

00:38.000 --> 00:41.000 Today, we are thrilled to have Guarav Chadha joining us.

00:41.000 --> 00:51.000 Guarav is a Senior Development Manager leading development of MySQL HeatWave Lakehouse with a keen interest in systems, machine learning and computer architecture.

00:51.000 --> 01:10.000 Guarav brings a multifaceted expertise to database technology. Following the completion of his PhD from the University of Michigan, Ann Arbor, Guarav started at Oracle Labs in 2016, working on a research project which eventually graduated into MySQL HeatWave.

01:10.000 --> 01:16.000 But today we will talk with him about MySQL and AI on premise. Welcome Guarav.

01:16.000 --> 01:17.000 Thanks, Fred. Hi, Scott.

01:17.000 --> 01:18.000 Hi, Guarav. How are you?

01:18.000 --> 01:19.000 Doing good.

01:19.000 --> 01:32.000 So we're going to dive right in. And AI, we see AI is taking over the world. It's being touted for the solution to everything.

01:32.000 --> 01:41.000 How do you see AI transforming traditional on-premise database environments, especially in enterprise setups?

01:41.000 --> 01:54.000 Yes, Scott. So, I completely agree. AI is a transformational technology, and it has the potential to improve everything that we see around us.

01:54.000 --> 02:07.000 So, with regards to traditional on-premise database environments, especially in enterprise setups, I see multiple categories here. So, AI is a technology and a toolset.

02:07.000 --> 02:32.000 And like many other operators in databases, it can help with more and different data analysis. So, think of AI as a new set of SQL operators, which can tease out or analyze data and derive insights that are hard to do it with other operators, with other analysis tools.

02:32.000 --> 02:45.000 And hard for folks to call up. And hard for folks to code up. And that's where I think AI enhances it very easily enters into the database environments.

02:45.000 --> 02:56.000 What I mean by that is examples are recommendation systems, anomaly detection, so on and so forth.

02:56.000 --> 03:02.000 The other category is what I would say user assistance.

03:02.000 --> 03:15.000 So, not everyone is a SQL expert. And we want database technology and databases to be accessible to more people who may or may not come from a traditional database background.

03:15.000 --> 03:22.000 And SQL is a very powerful language and where it can be daunting to start with.

03:22.000 --> 03:35.000 So, again, this is a general category where maybe folks who are not very familiar with a specific programming language like SQL could write things out in just plain natural text.

03:35.000 --> 03:42.000 And AI tools could translate this into a programmatic interface or programmatic language or SQL directly.

03:42.000 --> 03:50.000 And that's another facet where I think AI can make database systems more approachable to a larger category of folks.

03:50.000 --> 03:57.000 It can also give you more user friendly responses, like instead of saying, oh, here's the error code, something went wrong.

03:57.000 --> 04:00.000 It can give you more information, more user friendly responses.

04:00.000 --> 04:06.000 So those are some examples of where I would say the second category, user assistance.

04:06.000 --> 04:12.000 The third category of where AI could help is database management.

04:12.000 --> 04:21.000 So databases are systems of record, the sources of truth and have a very high bar of staying up and being available.

04:21.000 --> 04:30.000 AI can help schedule maintenance at the right time where maybe the workload is low.

04:30.000 --> 04:35.000 They can predict things that might get slow.

04:35.000 --> 04:43.000 We have a whole area called predictive maintenance and make databases more highly available, more easily approachable.

04:43.000 --> 04:44.000 Thank you.

04:44.000 --> 04:46.000 This sounds very interesting.

04:46.000 --> 04:50.000 And because we are talking about MySQL on-prem, right?

04:50.000 --> 05:04.000 So from these categories, what features could we expect then one day to see in MySQL enterprise with AI or for AI?

05:04.000 --> 05:07.000 So what can you tell us about that?

05:07.000 --> 05:16.000 So for MySQL AI, we are bringing a whole host of AI features to on-premise MySQL deployments.

05:16.000 --> 05:22.000 And we will lean heavily with this first version on the first category, which is data analysis.

05:22.000 --> 05:25.000 How can AI help with data analysis?

05:25.000 --> 05:34.000 And within this, I would focus on, I would say, a few subcategories.

05:34.000 --> 05:48.000 The first is, with AI and generative AI specifically, it has brought the industry a new tool set to search through and understand documents.

05:48.000 --> 05:58.000 And not just structured data or relational data, just plain documents, which is true for a lot of enterprise companies.

05:58.000 --> 06:09.000 Companies have years and years worth of information stored in documents, in PDF documents, in HTML documents, and not really put into a database necessarily.

06:09.000 --> 06:12.000 And this has always been hard to search.

06:12.000 --> 06:23.000 It has been very manual, it has been very hard to bring to a database and perform a very fast and meaningful search.

06:23.000 --> 06:39.000 With generative AI and what we call vector store and vector search, you can search through unstructured data like documents, semantically, instead of just through keywords.

06:39.000 --> 06:42.000 You can search them by meaning.

06:42.000 --> 06:47.000 That's a very powerful technology that we are bringing to MySQL AI.

06:47.000 --> 07:02.000 So if users have documents in their file systems, they can ingest them into the database, and we will automatically create what we call a vector store out of it, which prepares the data in these documents to be searched semantically.

07:02.000 --> 07:13.000 Obviously, in order to this, we are adding a new operator, which does this semantic search, we call this vector distance.

07:13.000 --> 07:31.000 Additionally, I spoke about data analysis tools like recommendation systems, like anomaly detection, and these operators also being brought to MySQL AI, where you can plug them into your logs, or you can plug them into other metrics.

07:31.000 --> 07:40.000 And figure out when things can go wrong, or any other domain that is useful.

07:40.000 --> 07:46.000 An example of a domain for anomaly detection would be financial fraud, credit card fraud.

07:46.000 --> 07:49.000 So it's very useful in those scenarios.

07:49.000 --> 07:56.000 And the last category I would say among data analysis is generative AI.

07:56.000 --> 08:15.000 We're bringing LLMs to MySQL AI, and the power of LLMs really is they can generate new data and new user-friendly text from just bullet points, for instance.

08:15.000 --> 08:22.000 So not just analyzing data, but generating new data is possible through LLMs.

08:22.000 --> 08:29.000 So that I would say covers the first category.

08:29.000 --> 08:31.000 This is all data analysis.

08:31.000 --> 08:34.000 Among the second category, which is user assistance.

08:34.000 --> 08:41.000 User assistance is by bringing LLMs to on-premise MySQL AI deployments.

08:41.000 --> 08:50.000 It gives the user freedom to build more user-friendly applications or make the existing applications more user-friendly.

08:50.000 --> 08:53.000 And this is what we will start with, with version one of MySQL AI.

08:53.000 --> 09:04.000 So are there any specific features in MySQL Enterprise, like Firewall or Enterprise Audit, that support AI-enabled applications?

09:04.000 --> 09:09.000 So as we discussed, AI is an incredibly powerful set of tools and technologies.

09:09.000 --> 09:15.000 And this is our first salvo in enabling our customers to build and augment applications using AI.

09:15.000 --> 09:28.000 So we're bringing a whole tool set, we're bringing faster data analysis, more meaningful and different kinds of data analysis to help users build and augment existing applications.

09:28.000 --> 09:34.000 The door is certainly open to bringing AI to the ecosystem of products, as you mentioned, around the MySQL server.

09:34.000 --> 09:39.000 But with this first version, we are building these right into the MySQL server.

09:39.000 --> 09:54.000 With this MySQL AI, like you call it, right, is it compatible with or will it be compatible with all the architecture solutions that we also provide on-premise,

09:54.000 --> 10:01.000 like such as the InnoDB cluster, the cluster set, replica set, you know, for HA, for disaster recovery?

10:01.000 --> 10:08.000 If somebody goes in that direction, will he be able to keep deploying his MySQL the same way?

10:08.000 --> 10:15.000 So the AI feature set, the tool sets, are built right into the MySQL server.

10:15.000 --> 10:23.000 So all the architecture solutions which are deploying MySQL AI instances benefit from these features.

10:23.000 --> 10:33.000 Okay, so just to be clear, so we're not going to have two distinct products where one's MySQL EE and another one is MySQL AI.

10:33.000 --> 10:35.000 They'll all be together in the same product?

10:35.000 --> 10:41.000 So MySQL AI will be a distinct offering.

10:41.000 --> 10:48.000 It will have everything that Enterprise Edition has, plus the additional AI features we spoke about.

10:48.000 --> 10:51.000 And this is all about user choice.

10:51.000 --> 10:57.000 Customers can continue using MySQL EE if that is what they prefer.

10:57.000 --> 11:10.000 They can switch to MySQL AI and or buy new MySQL AI deployments to try out these new AI tools and get them familiar, get themselves familiar with it.

11:10.000 --> 11:17.000 The MySQL AI will have everything that Enterprise Edition has, plus the AI features.

11:17.000 --> 11:32.000 So it opens the door for users and customers to build their applications worry-free, as they've always built with MySQL EE, because the entire feature set of EE will be present in MySQL AI.

11:32.000 --> 11:37.000 But additionally, they can build more newer things with AI.

11:37.000 --> 11:43.000 In the previous episode, so the other speakers or guests, right?

11:43.000 --> 11:49.000 They extol the virtues of the cloud for AI, our cloud.

11:49.000 --> 11:52.000 Everything was nice and fast and it's good.

11:52.000 --> 12:06.000 And I would like to ask you if there are a performance trade-off between then deploying AI solutions in the cloud using MySQL HeatWave versus on-prem with this new MySQL AI.

12:06.000 --> 12:21.000 So with MySQL AI, we have brought the AI technology, we have built and deployed in the cloud to our EE customers, to on-premise environments.

12:21.000 --> 12:24.000 Cloud has some unquestionable advantages.

12:24.000 --> 12:28.000 Cloud has the benefit of scale-out, which can bring higher performance.

12:28.000 --> 12:37.000 It has GPUs, which can execute LLMs faster or larger LLMs, higher quality LLMs.

12:37.000 --> 12:48.000 So with MySQL AI, we have brought the AI technology that we built and deployed in the cloud to our on-premise customers and our Enterprise Edition customers.

12:48.000 --> 12:57.000 There are a very large, very large set of customers who are on-premise for a variety of reasons.

12:57.000 --> 13:02.000 And we want to serve them where they are.

13:02.000 --> 13:07.000 What I do want to point out is that cloud has some unquestionable advantages.

13:07.000 --> 13:14.000 It has the benefit of scaling out with more and more hardware, which can give you high performance.

13:14.000 --> 13:22.000 And like HeatWave, all these AI features are built to scale out with more resources.

13:22.000 --> 13:31.000 Cloud also has GPUs, which bring more performance for LLMs and can execute larger and higher quality LLMs.

13:31.000 --> 13:47.000 Cloud also has our HeatWave analytics engine, which provides faster analytics performance, allowing users to build combined applications with analytics, OLTP, AI.

13:47.000 --> 13:58.000 What is very important to note, all AI features we bring into MySQL AI are 100% API compatible with HeatWave in the cloud.

13:58.000 --> 14:02.000 So users can build their applications on MySQL AI.

14:02.000 --> 14:15.000 And should they feel the need for higher performance or an expanded feature set, and they want to move to the cloud, the applications will work without modifications.

14:15.000 --> 14:19.000 And we have tried very hard to make it 100% API compatible.

14:19.000 --> 14:40.000 So with MySQL AI, we have optimized inference of open-width LLMs on CPUs, right from one core all the way to 192 cores, using proprietary weight caching and quantization techniques.

14:40.000 --> 14:49.000 And this allows us to, the users to deploy MySQL AI on a range of computer infrastructure, depending on their need.

14:49.000 --> 14:51.000 It can be very small MySQL node.

14:51.000 --> 14:53.000 It can be a very beefy MySQL node.

14:53.000 --> 15:13.000 And as they improve the hardware, the performance, the latency, the quality, the concurrency we deliver increases, providing incentive to users to deploy on larger and larger hardware.

15:13.000 --> 15:25.000 Are we targeting, like, is MySQL AI targeted towards existing MySQL EE customers who are looking for more features?

15:25.000 --> 15:35.000 Or are we kind of targeting other potential customers and luring them in with AI to get them into the MySQL ecosystem?

15:35.000 --> 15:50.000 Both. We are bringing more features to MySQL EE, and we hope that more and more users, more classes of users, more classes of applications, find their home in MySQL.

15:50.000 --> 16:00.000 And absolutely, we want to allow our existing on-premise customers to be able to bring more of their workloads into MySQL.

16:00.000 --> 16:06.000 And for a number of reasons, many customers want to be on-premise.

16:06.000 --> 16:10.000 They might require deployments in the edge devices.

16:10.000 --> 16:13.000 They might require deployments in air-gapped environments for data security.

16:13.000 --> 16:21.000 So we want to enable these existing customers to bring more workloads.

16:21.000 --> 16:29.000 Of course, there are other customers who may not have ever looked at MySQL EE because they have requirements to deploy AI or to deploy generative AI.

16:29.000 --> 16:41.000 And we want to obviously give them this tool set and this enhanced feature set to bring their primary, secondary, tertiary workloads to MySQL EE.

16:41.000 --> 16:46.000 That's very nice to bring this to our on-prem customer.

16:46.000 --> 17:05.000 So correct me if I'm wrong, but to what I understood is that we modified LLMs to work on normal CPU for performance, and so we can run it.

17:05.000 --> 17:20.000 We don't need to have GPUs, but you also said that, yeah, if we really want to use a very large LLM, then it's better to use it to use the cloud than on-prem.

17:20.000 --> 17:33.000 But I wanted to also ask you, because LLMs that are evolving and not at the same speed of the MySQL releases.

17:33.000 --> 17:35.000 I have two questions in this one.

17:35.000 --> 17:42.000 It's like, oh, if there are new LLMs, will it be the possibility to the user to use it directly in GenAI?

17:42.000 --> 17:49.000 Or will it be updated at every new release of MySQL?

17:49.000 --> 18:01.000 Good question. So there's absolutely no exaggeration to state that a new LLM seemingly is released every month with its unique set of characteristics and benefits.

18:01.000 --> 18:20.000 So we will definitely bring new LLMs, which enhance the performance of all quality of results, as and when we feel that is useful to add to the existing set of LLMs we offer with future MySQL releases.

18:20.000 --> 18:31.000 So we will have we have taken the approach of building in LLMs in the package that we ship to our customers.

18:31.000 --> 18:37.000 There are many reasons for this. We have optimized these LLMs to run on CPUs.

18:37.000 --> 18:54.000 So we are able to run larger LLMs faster on CPUs, allowing our users to seamlessly use these LLMs without extra hardware or call outs to other services.

18:54.000 --> 19:02.000 So we will continue doing that with new LLMs as new ones prove to be useful for our users.

19:02.000 --> 19:14.000 What you say is that, OK, when we're going to release a new version of MySQL, if there are new LLMs that were interesting to update, they will be updated at that time, right?

19:14.000 --> 19:19.000 Build the LLMs into the package the users download and deploy.

19:19.000 --> 19:24.000 Users do not need to bring their own LLMs.

19:24.000 --> 19:51.000 Gurav, thank you for joining us today. It really was interesting for me, I can say personally, to learn about some of the advancements that are coming in MySQL AI and how it's going to be integrating with our EE version to allow people who need on premise or prefer on premise installations of MySQL to actually harness some of the AI power that we offer in HeatWave.

19:51.000 --> 19:54.000 Thank you, Scott. Thank you, Fred.

19:54.000 --> 19:55.000 Thank you very much. Bye bye.

19:55.000 --> 20:00.000 That's a wrap on this episode of Inside MySQL: Sakila Speaks. Thanks for hanging out with us.

20:00.000 --> 20:04.000 If you enjoyed listening, please click subscribe to get all the latest episodes.

20:04.000 --> 20:07.000 We would also love your reviews and ratings on your podcast app.

20:07.000 --> 20:12.000 Be sure to join us for the next episode of Inside MySQL: Sakila Speaks.