On openvpn setups where MD5 is still in use with the SSL certificate, it’s now impossible to connect when you are in Fedora 21.
To allow openvpn client to connect to such VPN using NetworkManager, you need to modify /usr/lib/systemd/system/NetworkManager.service and add the following two lines in [Service] section:
Environment="NSS_HASH_ALG_SUPPORT=+MD5" Environment="OPENSSL_ENABLE_MD5_VERIFY=1"
Restart the service and it will work 
Hello,
Like every year since I joined Percona, I also plan to speak at the next Percona Live Conference & Expo in Santa Clara during spring 2016.
Once again I want to share with the users, DBAs and developers my experience related to MySQL High Availability and especially with Galera.
This year, Percona implemented a Community Vote to rate the talks you would like to see in the schedule. So if you want to attend one or more of my talks, please vote for them.
Of course, I invite you to rate other people’s talk too, that will help the organisation and the Conference Committee to prepare the schedule.
These are the direct links to my talks and tutorial:
Hi, I’ve finally decided to upgrade my old drupal 6.x blog… I tried Drupal 8, but to be honest I finally migrated to WordPress.
WordPress is more easy to maintain, to deal with and also it has a nice Mobile App that I can use to review comments, edit posts, etc…
Welcome to the new blog and let’s hope I will have more time to add nice stuff here
At Percona Live 2016, I presented a talk related to Galera. I tried to explain what’s really behind the magic. I think it’s essential for all who want to use Galera replication in MySQL to understand all these concepts.
These are the slides:
Also during Percona Live with my colleague Kenny Gryp, we delivered a tutorial on Percona XtraDB Cluster (Galera)
These are the slides:
As some already noticed it, I’ve recently joined the amazing MySQL family in Oracle.
I’m now part to the awesome MySQL Community Team and I will be mostly present in EMEA.
Why did I make such change ? Oracle offered me the opportunity to work in a domain I really like and I was already trying to represent the MySQL Community as much as I could but mostly in using my own free time (twitter, facebook, FOSDEM, …). Now this is my full time job !
Yes, OK, but why Oracle ? To be honest this answer is very easy to reply. Somebody interested in the MySQL Community for the last few years should be blind to not see the amount of work the MySQL engineering team is doing: many improvements and overall, many new features. I won’t be a lie if I say that Oracle’s MySQL is also the most active in innovating MySQL (JSON data type, X plugin, MySQL shell,… )
For all these reasons, I’m very proud to have made the choice to join this fantastic adventure and I’m already looking forward to meet many Community members, users, developers and share with you all the new features we release, discover and take in consideration your needs and remarks.
As Morgan announced it his blog post related to new defaults in 5.7, in 5.7.7 and newer the default value of sync_binlog is now 1.
This of course has an impact on performance especially if you don’t have fast storage like a RAID controller with a cache for example.
Already in O’reilly High Performance MySQL, is was written that the most important setting for binary logging on the master is sync_binlog = 1.
And this recommendation was made for MySQL 5.0 ! At that time the performance impact could reach 50%. In 5.6, with the addition of binary log group commit the impact from enabling sync_binlog = 1 on many non-synthetic workloads should be reduced considerably and is the safest option as all transactions are synchronized to the binary log before they are committed. Therefore, even after a crash, any transactions that are missing from the binary log are only in prepared state and those transaction will be rollback during the recovery process at server restart. This guarantees that no transaction is lost from the binary log, and is the safest option. If disabled you could have some transaction that have been committed but not present in the binary log. This could be very dangerous in some architecture were more and more people rely on replication. For example if you provision a new slave using last night’s backup and replay all binary logs since, the master and the slave won’t be in sync.
This is a major change compare to previous versions and don’t forget that it needs innodb_support_xa to be enabled (this is the default). So yes, since 5.6 during recovery the binary logs can be edited !
So as conclusion:
Just for information, the current documentation, doesn’t seem to be 100% accurate. I entered a bug (#81800) to review it.
Last week during the Oracle Users Group Leaders Summit in Bucharest I had the pleasure the meet the leaders for the MySQL Users Group from Azerbaijan, Finland, Madrid and the Netherlands.
During some discussions, it appeared that some users are not aware of the Performance_Schema integration in MySQL Workbench. Indeed with WB you can enable PFS, add sys table if not present by default (<5.7).
You can use a default configuration or customize it and enable all the instruments and consumers you need.
When enabled, you can generate some useful reports
In case you are also not yet aware of it, the following dashboard is also available in MySQL Workbench
The latest version of MySQL Workbench has been released yesterday, June 13th : 6.3.7 GA
From September 18–22, 2016 in San Francisco will be held the 2016’s edition of Oracle Open World. The MySQL Team will be well represented with more than 20 sessions related to all the new stuff we released for 5.7 and many other amazing things for 8.0.
I have the honor to present you a tutorial on one of the most wanted feature: Group Replication.
MySQL Group Replication is a new plugin that ensures virtual synchronous updates on any member in a group of MySQL servers, with conflict handling and failure detection. Distributed recovery is also in the package to ease the process of adding new servers to your server group.
This tutorial will be full hands-on. Matt Lord and myself will guide on the process of upgrading your asynchronous replication architecture to the new multi-master group replication !
This is the link to our tutorial: TUT6394
And this is the list of all MySQL related sessions.
I hope to see you there !
Two weeks after the big annual event in San Francisco, Oracle Open World 2016 , September 18-22, there will be another MySQL focused event in Europe: Percona Live Amsterdam, October 3-5.
The MySQL Engineering Team and Community Team will be present. We have submitted a list of very interesting talks, mostly related to the upcoming new major release of MySQL.
In an effort to involve the larger community in the talks selections, PLAM’s organization implemented a community voting process for the previous edition. If this will be again the case and if you are interested by our sessions, don’t hesitate to vote, if the vote is available, for the sessions you would like to see or just tweet about them using #perconalive
Here is the list of the talks we submitted:
As you can see, we tried to cover all the main topics and focusing on our new key features like the new support for CTEs, the new data dictionary, Group Replication, using MySQL as a document store with the X Protocol and many other improvements for the future MySQL 8.0 !
As announced by Luis Soares on MySQL High Availability’s blog, a new labs version of Group Replication (GR) has been released for MySQL 5.7.14 !
Group Replication is the main component of the future MySQL HA solution. Even if with GR, it’s possible to write simultaneously on all the member of the group, GR doesn’t provide any write scaling. Indeed, if one member reached 100% of its write capabilities, adding writes to another member, won’t help as both will have to write their load AND the load from the other member: remember this is (virtual) synchronous replication, all writes happening on member A will happen on member B and on all other members of the group.
As you can read above, Group Replication has its own terminology:
A quick start guide has been published by Matt Lord: here.
This article is the first of a new series related to Group Replication.
Don’t forget that the MySQL Team will propose several sessions on GR at Oracle Open World 2016 (OOW):
I will also present an hands-on tutorial on Group Replication at Percona Live Amsterdam with my friend Kenny Gryp.
And finally, don’t miss the awesome articles from the development team on mysqlhihavailability’s blog.
From October 3rd to 5th, the MySQL Engineering and Community Team will be in Amsterdam to present MySQL 8.0 at Percona Live.
The Conference Committee has rated all the numerous submissions and I’m very proud to announce that we will deliver the following 11 sessions:
I hope to meet you there
Vadim Tkachenko recently released Docker images for Group Replication (thank you for that ).
As not everybody is already using Docker or just because having multiple choices is also nice (this is an OpenSource world isn’t it ?), I decided to share a Vagrant environment that you can use to evaluate Group Replication.
This environment provides also a module that can be used to deploy GR on any other environment managed by Puppet.
The repository is available on my github account: mysqlGR-vagrant
This is a quick demo on how to use it:
Tadam ! You have a MySQL Group of 3 members ready to use !
MySQL Group Replication is just one component of the upcoming MySQL HA Solution as Matt Lord explained it in his GR quick start guide. So while mysql-router is back to school to become smarter and learn how to take the right decision we can already use existing 3rd party solutions. To be honest, if people are already using a load balancer/proxy with MySQL asynchronous, semi-sync or other virtual synchronous solution and want to evaluate and migrate to MySQL’s native Group Replication, they might not migrate load balancer at the same time as they move to GR. Step-by-step migration, one component at the time is always recommended in complex architecture.
So in this post, I want to cover HAProxy and a dedicated health check related to what might be the more common architecture. Matt already explained in his start guide how to use GR with HA. He also provided and healthcheck scripts and a HAProxy configuration when using only one member (recommended for WRITEs) and the other ones stay as backup.
What is then that most common architecture ?
So due to the GR limitations (DDL, FK, …) it is recommended to send the WRITEs to one node (unless you really know your workload and what you are doing). For the READs, we can use any of the group members (unless they still have a large queue to process).
Usually people use then 2 different ports, one to send the WRITEs and one to send the READs. Of course the application should be able to define which type of query is needed and use the right connection (db handler).
This is an illustration of the architecture:
So HAProxy is listening on :3307 to handle all writes and send them only on one dedicated member. Failover for writes will happen only is MySQL has a problem or becomes read-only. Reads are sent to port :3308 and HAProxy will round-robin those reads to all members of the group.
This is the HAProxy configuration:
frontend mysql-gr-front_write bind *:3307 mode tcp default_backend mysql-gr-back_write backend mysql-gr-back_write mode tcp balance leastconn option httpchk server mysql1 192.168.90.2:3306 check port 6446 inter 1000 rise 1 fall 2 on-marked-up shutdown-backup-sessions server mysql2 192.168.90.3:3306 check port 6446 inter 1000 rise 1 fall 2 backup server mysql3 192.168.90.4:3306 check port 6446 inter 1000 rise 1 fall 2 backup frontend mysql-gr-front_read bind *:3308 mode tcp default_backend mysql-gr-back_read backend mysql-gr-back_read mode tcp balance leastconn option httpchk server mysql1 192.168.90.2:3306 check port 6447 inter 1000 rise 1 fall 2 server mysql2 192.168.90.3:3306 check port 6447 inter 1000 rise 1 fall 2 server mysql3 192.168.90.4:3306 check port 6447 inter 1000 rise 1 fall 2
We will then setup 2 different xinetd services to check the status of the MySQL Group Member and decide if the node can be used for reads and or writes.
To perform such check, I’m using Matt Lord’s script that I modified to accept two arguments:
This script is available on github : https://github.com/lefred/mysql_gr_routing_check
We have then:
/etc/xinetd.d/mysql_gr_routing_check_write
# default: on
# description: check to see if the node is a viable routing candidate
service mysql_gr_routing_check_write
{
disable = no
flags = REUSE
socket_type = stream
port = 6446
wait = no
user = mysql
server = /usr/local/bin/mysql_gr_routing_check.sh
server_args = 100 write
log_on_failure += USERID
only_from = localhost 192.168.90.0/24
per_source = UNLIMITED
}
/etc/xinetd.d/mysql_gr_routing_check_read
# default: on
# description: check to see if the node is a viable routing candidate
service mysql_gr_routing_check_read
{
disable = no
flags = REUSE
socket_type = stream
port = 6447
wait = no
user = mysql
server = /usr/local/bin/mysql_gr_routing_check.sh
server_args = 100 read
log_on_failure += USERID
only_from = localhost 192.168.90.0/24
per_source = UNLIMITED
}
Don’t forget to update /etc/services with:
mysql_gr_routing_check_write 6446/tcp # MySQL Group Replication mysql_gr_routing_check_read 6447/tcp # MySQL Group Replication
So now we can test the health check on both ports:
[root@mysql1 bin]# telnet 192.168.90.2 6446 Trying 192.168.90.2... Connected to 192.168.90.2. Escape character is '^]'. HTTP/1.1 200 OK Content-Type: text/plain Connection: close Content-Length: 40 Group Replication member is a viable routing candidate for write. Connection closed by foreign host. [root@mysql1 bin]# telnet 192.168.90.2 6447 Trying 192.168.90.2... Connected to 192.168.90.2. Escape character is '^]'. HTTP/1.1 200 OK Content-Type: text/plain Connection: close Content-Length: 40 Group Replication member is a viable routing candidate for read. Connection closed by foreign host.
Perfect 
This is an illustration of this setup in HAProxy Web frontend:
Let’s check this in action:
After having played with MySQL Group Replication and HAProxy, it’s time to show you how easy it’s to setup MySQL HA with ProxySQL.
ProxySQL is a high performance open source proxy for MySQL. It has many features that invite you to discover on proxysql.com and on github.
If you remember, I wrote in my last post that it is recommended to use Group Replication with only one WRITER group member. As it is the preferred architecture, I will show you how to achieve this using ProxySQL. With ProxySQL, you don’t need to have two different interfaces to split reads and writes.
In fact, when you use ProxySQL, you have a much larger amount of options to route your queries. In production, the smart DBA, will identify the queries that would be better to move away from the writer member. ProxySQL allows you to route the queries using regexp (this is what I am using here for demo purpose) but also digest of your queries. This is what I would recommend to use in production.
So in summary about how to define the best routing, you should configure ProxySQL to send all the traffic to only one member. Then check and determine which are the expensive read statements in “`stats_mysql_query_digest“` and create rules to send those queries using their digest to the READ hostgroup.
Let’s go back to our architecture for today. This is how it looks like :
So we have two hostgroups :
Let me show you how to configure those groups in ProxySQL:
[root@mysql1 ~]# mysql -u admin -padmin -h 127.0.0.1 -P 6032 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 0 Server version: 5.5.30 (ProxySQL Admin Module) Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql1 mysql> \R Admin> PROMPT set to 'Admin> ' Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'192.168.90.2',3306); Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'192.168.90.3',3306); Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'192.168.90.4',3306); Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (2,'192.168.90.2',3306); Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (2,'192.168.90.3',3306); Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (2,'192.168.90.4',3306);
So here we can see we will use the same Groups members that are part of our “cluster”.
Admin> select * from mysql_servers; +--------------+--------------+------+--------------+--------+-------------+-----------------+---------------------+---------+----------------+ | hostgroup_id | hostname | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | +--------------+--------------+------+--------------+--------+-------------+-----------------+---------------------+---------+----------------+ | 1 | 192.168.90.2 | 3306 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | 1 | 192.168.90.3 | 3306 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | 1 | 192.168.90.4 | 3306 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | 2 | 192.168.90.2 | 3306 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | 2 | 192.168.90.3 | 3306 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | | 2 | 192.168.90.4 | 3306 | ONLINE | 1 | 0 | 1000 | 0 | 0 | 0 | +--------------+--------------+------+--------------+--------+-------------+-----------------+---------------------+---------+----------------+
Easy isn’t it ?
Now we will add a scheduler that will use a script that verifies our MySQL InnoDB Cluster (Group Replication). The script is available on my github: here and you can put it in “`/var/lib/proxysql/“`
Admin> INSERT INTO scheduler(id,interval_ms,filename,arg1,arg2,arg3,arg4, arg5)
VALUES (1,'10000','/var/lib/proxysql/proxysql_groupreplication_checker.sh','1','2','1','0','/var/lib/proxysql/proxysql_groupreplication_checker.log');
Let’s save and load the scheduler:
Admin> SAVE SCHEDULER TO DISK; Admin> LOAD SCHEDULER TO RUNTIME;
What are those values in arg1 to arg5 ?
So now we can see that the script modified the status of the members :
Admin> select * from mysql_servers; +--------------+--------------+------+--------------+--------+-------------+-----------------+---------------------+---------+----------------+ | hostgroup_id | hostname | port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | +--------------+--------------+------+--------------+--------+-------------+-----------------+---------------------+---------+----------------+ | 1 | 192.168.90.2 | 3306 | ONLINE | 1 | 0 | 1000 | 5 | 0 | 0 | | 1 | 192.168.90.3 | 3306 | OFFLINE_SOFT | 1 | 0 | 1000 | 5 | 0 | 0 | | 1 | 192.168.90.4 | 3306 | OFFLINE_SOFT | 1 | 0 | 1000 | 5 | 0 | 0 | | 2 | 192.168.90.2 | 3306 | OFFLINE_SOFT | 1 | 0 | 1000 | 10 | 0 | 0 | | 2 | 192.168.90.3 | 3306 | ONLINE | 1 | 0 | 1000 | 10 | 0 | 0 | | 2 | 192.168.90.4 | 3306 | ONLINE | 1 | 0 | 1000 | 10 | 0 | 0 | +--------------+--------------+------+--------------+--------+-------------+-----------------+---------------------+---------+----------------+
It’s time to add some routing rules to be able to use those hostgroups. If you don only the first hostgroup will be used.
Admin> insert into mysql_query_rules (active, match_pattern, destination_hostgroup, apply) values (1,"^SELECT",2,1);
We will route all queries starting by select (this is not a recommendation of course a we will also send to hostgroup 2 all SELECT… FOR UPDATE, for example).
Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
You can now save to disk the setup as it works as expected and it was easy and quick to setup:
Admin> save MYSQL SERVERS TO DISK; Admin> SAVE MYSQL QUERY RULES TO DISK;
This is a demo in video :
Yesterday I was in Paris to attend a OpenTech Meetup related to MySQL.
You can find below the two presentations I gave (Warning: in French).
The audience was very interested and I got several good questions. It seems those two topics were very new for the majority of the developers, sysadmins and DBAs even if they are using MySQL daily.
This is a small demo of MySQL Shell as I couldn’t show it live.
Please note that if you were already using MySQL Shell in Python before 1.0.5, the names of the functions were different.
And finally, thank you to my colleague Oliver Dasini who posted some pics on twitter
During Oracle OpenWorld, on Tuesday, September 20th, at 7.00pm, the place to be will be at the MySQL Community Reception !
OpenWorld attendees, members of local MySQL user groups, MySQL users in the Bay Area – you’re all invited to Oracle’s MySQL Community Reception to meet the MySQL Team !
Mingle with your peers, run into old friends from around the MySQL Community and/or meet the MySQL engineers to discuss the new features of MySQL 8.0, discuss performance with DimitriK or Group Replication with the replication team !
And don’t forget to chat with the MySQL Oracle ACEs that will be present too.
You can already register to the event’s page!
We are looking forward to see you there.
Percona Live Amsterdam is just finished… there was an entire room full day about MySQL 8.0 which is wonderful but there was also another product that everybody was talking about: ProxySQL (great job René).
Ronald Bradford and the MySQL engineering team were discussing about what tool was missing in the MySQL ecosystem and Mark Leith highlighted that a datamasking utility was really missing.
Having used already ProxySQL for Group Replication and other tests, I knew that it would be possible to use it also for this purpose. After a very short discussion and a nightly chat with René, this is an example on how you can achieve this missing (and sometime heavy) task.
Description:
So we decided this:
This is our table:
+----+-----------+-------------+------------+------------------+----------+ | id | firstname | lastname | cc_type | cc_num | cc_verif | +----+-----------+-------------+------------+------------------+----------+ | 1 | Frederic | Descamps | mastercard | 5275653223285289 | 456 | | 8 | Dim0 | Vanoverbeke | mastercard | 5345654523285289 | 123 | | 15 | Kenny | Gryp | visa | 4916066793184589 | 456 | +----+-----------+-------------+------------+------------------+----------+
So first we will create on our database server the developer’s user:
MySQL> GRANT SELECT, USE on myapp.* to 'devel'@'%' identified by 'password';
Now let’s configure ProxySQL (I assume here that you already have installed and started proxysql) from it’s admin interface:
# mysql -u admin -padmin -h 127.0.0.1 -P6032
The first step is to create a user here too:
ProxySQL> INSERT INTO mysql_users(username,password,default_hostgroup)
VALUES ('devel','password',1);
and now our MySQL server if it's not yet referenced:
ProxySQL> INSERT INTO mysql_servers(hostgroup_id,hostname,port)
VALUES (1,'mysql1',3306);
We can load those settings in runtime and then save then to disk too:
ProxySQL> LOAD MYSQL SERVERS TO RUNTIME; ProxySQL> LOAD MYSQL USERS TO RUNTIME; ProxySQL> SAVE SERVERS USERS TO DISK; ProxySQL> SAVE MYSQL USERS TO DISK;
We can already try and see that our developer can still read our data. Let’s work on this now.
ProxySQL> INSERT INTO mysql_query_rules (rule_id,active,username,match_pattern,error_msg)
VALUES (90,1,'devel','^SELECT \*.*FROM.*CUSTOMERS',
'Query not allowed due to sensitive information, please contact dba@myapp.com');
Let’s load it in runtime and test:
ProxySQL> LOAD MYSQL QUERY RULES TO RUNTIME; # mysql -u devel -ppassword -h 127.0.0.1 -P 6033 mysql> select * from myapp.CUSTOMERS; ERROR 1148 (42000): Query not allowed due to sensitive information, please contact dba@myapp.com
Wooohooo it worked
And the datamasking ? This is our next step, let’s enter the following rule:
ProxySQL> INSERT INTO mysql_query_rules (rule_id,active,username,match_pattern,replace_pattern,apply)
VALUES (1,1,'devel','^[sS][eE][lL][eE][cC][tT] (.*)cc_num([ ,])(.*)',
"SELECT \1CONCAT(REPEAT('X',12),RIGHT(cc_num,4)) cc_num\2\3",1);
ProxySQL> LOAD MYSQL QUERY RULES TO RUNTIME;
We can test it again:
mysql> select firstname, cc_num from myapp.CUSTOMERS; +-----------+------------------+ | firstname | cc_num | +-----------+------------------+ | Frederic | XXXXXXXXXXXX5289 | | Dim0 | XXXXXXXXXXXX5289 | | Kenny | XXXXXXXXXXXX4589 | +-----------+------------------+
Wooohoooo it worked again
Note: if you find ways to workaround the datamasking, let me know, thank you.
Don’t forget to also save your rules to disk:
ProxySQL> SAVE MYSQL QUERY RULES TO DISK;
As you can see it’s possible to have a very quick datamasking solution for MySQL using ProxySQL.
First of all I want to say thanks to Thomas that sent me feedback on part1 (see his comment). And indeed all three cases were able to defeat the datamasking.
I rewrote then the rules to also take those cases in consideration:
ProxySQL> DELETE FROM mysql_query_rules where rule_id < 50;
ProxySQL> INSERT INTO mysql_query_rules
(rule_id,active,username,match_pattern,replace_pattern,apply)
VALUES (1,1,'devel','`cc_num`',"cc_num",0);
ProxySQL> INSERT INTO mysql_query_rules
(rule_id,active,username,match_pattern,replace_pattern,apply)
VALUES (2,1,'devel','^[sS][eE][lL][eE][cC][tT] (.*)cc_num([ ,\n])(.*)',
"SELECT \1CONCAT(LEFT(cc_num,2),REPEAT('X',10)) cc_num\2\3",1);
However the current recursive implementation of ProxySQL (using flagIN & flagOUT) is not an option to face the case where a masked field is referenced twice. To handle this case, ProxySQL will have to implement PCRE (see issue 716).
Additionally, in part 1, we saw how to mask one column for a specific user.
But if I want to mask other fields in the same table ? Is that feasible ? Easily ?
Yes and yes
The only necessarily steps are:
In the following example, I use the same table as in the previous article, but this time I want to also mask the last name, I decided to only show the first two character and use a fixed length.
ProxySQL> update mysql_query_rules set apply=0 where rule_id<3;
ProxySQL> INSERT INTO mysql_query_rules
(rule_id,active,username,match_pattern,replace_pattern,apply)
VALUES (4,1,'devel','`lastname`',"lastname",0);
ProxySQL> INSERT INTO mysql_query_rules
(rule_id,active,username,match_pattern,replace_pattern,apply)
VALUES (5,1,'devel','^[sS][eE][lL][eE][cC][tT] (.*)lastname([ ,\n])(.*)',
"SELECT \1CONCAT(LEFT(lastname,2),REPEAT('X',10)) lastname\2\3",1);
ProxySQL> LOAD MYSQL QUERY RULES TO RUNTIME;
Et voilà, done !
Let’s test it:
mysql1 mysql> select firstname, cc_num, lastname from myapp.CUSTOMERS; +-----------+------------------+--------------+ | firstname | cc_num | lastname | +-----------+------------------+--------------+ | Frederic | XXXXXXXXXXXX5289 | DeXXXXXXXXXX | | Dim0 | XXXXXXXXXXXX5289 | VaXXXXXXXXXX | | Kenny | XXXXXXXXXXXX4589 | GrXXXXXXXXXX | +-----------+------------------+--------------+
Wooohoooo it worked !
With ProxySQL you can also easily check which rules have been hit and how many times:
ProxySQL> SELECT hits, mysql_query_rules.rule_id, match_digest,
substr(match_pattern,1,20) "match",
substr(replace_pattern,1,20) "replace", apply
FROM mysql_query_rules
NATURAL JOIN stats.stats_mysql_query_rules
ORDER BY mysql_query_rules.rule_id;
+------+---------+--------------+----------------------+----------------------+-------+
| hits | rule_id | match_digest | match | replace | apply |
+------+---------+--------------+----------------------+----------------------+-------+
| 2 | 1 | NULL | `cc_num` | cc_num | 0 |
| 2 | 2 | NULL | ^[sS][eE][lL][eE][cC | SELECT \1CONCAT(LEFT | 0 |
| 1 | 4 | NULL | `lastname` | lastname | 0 |
| 1 | 5 | NULL | ^[sS][eE][lL][eE][cC | SELECT \1CONCAT(LEFT | 1 |
| 0 | 90 | NULL | ^SELECT \*.*FROM.*CU | NULL | 0 |
+------+---------+--------------+----------------------+----------------------+-------+
So at the end, datamasking for MySQL is not so complicated.
Note: As Tom Krouper highlighted it to me by chat, it’s a VERY bad idea to store Credit Card numbers in clear into a database. This was just for the theory, but you have been warned !
The slides of my joint tutorial with Kenny Gryp from Percona on MySQL InnoDB Cluster and Group Replication are now online.
We got nice questions during the session and very good feedback.