4.4 The DataSource 6.0.0

(Quick Reference) # 4.4 The DataSource Version: 6.0.0 ## Table of Contents ## 4.4 The DataSource Since Grails is built on Java technology setting up a data source requires some knowledge of JDBC (the technology that stands for Java Database Connectivity). If you use a database other than H2 you need a JDBC driver. For example for MySQL you would need Connector/J. Drivers typically come in the form of a JAR archive. It’s best to use the dependency resolution to resolve the jar if it’s available in a Maven repository, for example you could add a dependency for the MySQL driver like this:
    dependencies {
        runtime 'mysql:mysql-connector-java:5.1.29'
    }
Once you have the JAR resolved you need to get familiar with how Grails manages its database configuration. The configuration can be maintained in either `grails-app/conf/application.groovy` or `grails-app/conf/application.yml`. These files contain the dataSource definition which includes the following settings: `driverClassName` - The class name of the JDBC driver `username` - The username used to establish a JDBC connection `password` - The password used to establish a JDBC connection `url` - The JDBC URL of the database `dbCreate` - Whether to auto-generate the database from the domain model - one of 'create-drop', 'create', 'update', 'validate', or 'none' `pooled` - Whether to use a pool of connections (defaults to true) `logSql` - Enable SQL logging to stdout `formatSql` - Format logged SQL `dialect` - A String or Class that represents the Hibernate dialect used to communicate with the database. See the org.hibernate.dialect package for available dialects. `readOnly` - If `true` makes the DataSource read-only, which results in the connection pool calling `setReadOnly(true)` on each `Connection` `transactional` - If `false` leaves the DataSource’s transactionManager bean outside the chained BE1PC transaction manager implementation. This only applies to additional datasources. `persistenceInterceptor` - The default datasource is automatically wired up to the persistence interceptor, other datasources are not wired up automatically unless this is set to `true` `properties` - Extra properties to set on the DataSource bean. See the Tomcat Pool documentation. There is also a Javadoc format documentation of the properties. `jmxExport` - If `false`, will disable registration of JMX MBeans for all DataSources. By default JMX MBeans are added for DataSources with `jmxEnabled = true` in properties. `type` - The connection pool class if you want to force Grails to use it when there are more than one available. A typical configuration for MySQL in `application.groovy` may be something like:
    dataSource {
        pooled = true
        dbCreate = "update"
        url = "jdbc:mysql://localhost:3306/my_database"
        driverClassName = "com.mysql.jdbc.Driver"
        dialect = org.hibernate.dialect.MySQL5InnoDBDialect
        username = "username"
        password = "password"
        type = "com.zaxxer.hikari.HikariDataSource"
        properties {
           jmxEnabled = true
           initialSize = 5
           maxActive = 50
           minIdle = 5
           maxIdle = 25
           maxWait = 10000
           maxAge = 10 * 60000
           timeBetweenEvictionRunsMillis = 5000
           minEvictableIdleTimeMillis = 60000
           validationQuery = "SELECT 1"
           validationQueryTimeout = 3
           validationInterval = 15000
           testOnBorrow = true
           testWhileIdle = true
           testOnReturn = false
           jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
           defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
        }
    }
| When configuring the DataSource do not include the type or the def keyword before any of the configuration settings as Groovy will treat these as local variable definitions and they will not be processed. For example the following is invalid:
    dataSource {
        boolean pooled = true // type declaration results in ignored local variable
        ...
    }
Example of advanced configuration using extra properties:
    dataSource {
        pooled = true
        dbCreate = "update"
        url = "jdbc:mysql://localhost:3306/my_database"
        driverClassName = "com.mysql.jdbc.Driver"
        dialect = org.hibernate.dialect.MySQL5InnoDBDialect
        username = "username"
        password = "password"
        type = "com.zaxxer.hikari.HikariDataSource"
        properties {
           // Documentation for Tomcat JDBC Pool
           // http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#Common_Attributes
           // https://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/tomcat/jdbc/pool/PoolConfiguration.html
           jmxEnabled = true
           initialSize = 5
           maxActive = 50
           minIdle = 5
           maxIdle = 25
           maxWait = 10000
           maxAge = 10 * 60000
           timeBetweenEvictionRunsMillis = 5000
           minEvictableIdleTimeMillis = 60000
           validationQuery = "SELECT 1"
           validationQueryTimeout = 3
           validationInterval = 15000
           testOnBorrow = true
           testWhileIdle = true
           testOnReturn = false
           ignoreExceptionOnPreLoad = true
           // http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#JDBC_interceptors
           jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
           defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED // safe default
           // controls for leaked connections
           abandonWhenPercentageFull = 100 // settings are active only when pool is full
           removeAbandonedTimeout = 120
           removeAbandoned = true
           // use JMX console to change this setting at runtime
           logAbandoned = false // causes stacktrace recording overhead, use only for debugging
           // JDBC driver properties
           // Mysql as example
           dbProperties {
               // Mysql specific driver properties
               // http://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html
               // let Tomcat JDBC Pool handle reconnecting
               autoReconnect=false
               // truncation behaviour
               jdbcCompliantTruncation=false
               // mysql 0-date conversion
               zeroDateTimeBehavior='convertToNull'
               // Tomcat JDBC Pool's StatementCache is used instead, so disable mysql driver's cache
               cachePrepStmts=false
               cacheCallableStmts=false
               // Tomcat JDBC Pool's StatementFinalizer keeps track
               dontTrackOpenResources=true
               // performance optimization: reduce number of SQLExceptions thrown in mysql driver code
               holdResultsOpenOverStatementClose=true
               // enable MySQL query cache - using server prep stmts will disable query caching
               useServerPrepStmts=false
               // metadata caching
               cacheServerConfiguration=true
               cacheResultSetMetadata=true
               metadataCacheSize=100
               // timeouts for TCP/IP
               connectTimeout=15000
               socketTimeout=120000
               // timer tuning (disable)
               maintainTimeStats=false
               enableQueryTimeouts=false
               // misc tuning
               noDatetimeStringSync=true
           }
        }
    }
#### More on dbCreate Hibernate can automatically create the database tables required for your domain model. You have some control over when and how it does this through the `dbCreate` property, which can take these values: create - Drops the existing schema and creates the schema on startup, dropping existing tables, indexes, etc. first. create-drop - Same as create, but also drops the tables when the application shuts down cleanly. update - Creates missing tables and indexes, and updates the current schema without dropping any tables or data. Note that this can’t properly handle many schema changes like column renames (you’re left with the old column containing the existing data). validate - Makes no changes to your database. Compares the configuration with the existing database schema and reports warnings. any other value - does nothing Setting the `dbCreate` setting to "none" is recommended once your schema is relatively stable and definitely when your application and database are deployed in production. Database changes are then managed through proper migrations, either with SQL scripts or a migration tool like Flyway or Liquibase. The Database Migration plugin uses Liquibase. ## 4.4.1 DataSources and Environments The previous example configuration assumes you want the same config for all environments: production, test, development etc. Grails' DataSource definition is "environment aware", however, so you can do:
    dataSource {
        pooled = true
        driverClassName = "com.mysql.jdbc.Driver"
        dialect = org.hibernate.dialect.MySQL5InnoDBDialect
        // other common settings here
    }

    environments {
        production {
            dataSource {
                url = "jdbc:mysql://liveip.com/liveDb"
                // other environment-specific settings here
            }
        }
    }
## 4.4.2 Automatic Database Migration The `dbCreate` property of the `DataSource` definition is important as it dictates what Grails should do at runtime with regards to automatically generating the database tables from GORM classes. The options are described in the DataSource section: `create` `create-drop` `update` `validate` no value In development mode `dbCreate` is by default set to "create-drop", but at some point in development (and certainly once you go to production) you’ll need to stop dropping and re-creating the database every time you start up your server. It’s tempting to switch to `update` so you retain existing data and only update the schema when your code changes, but Hibernate’s update support is very conservative. It won’t make any changes that could result in data loss, and doesn’t detect renamed columns or tables, so you’ll be left with the old one and will also have the new one. Grails supports migrations with Liquibase or Flyway via plugins. Database Migration Flyway ## 4.4.3 Transaction-aware DataSource Proxy The actual `dataSource` bean is wrapped in a transaction-aware proxy so you will be given the connection that’s being used by the current transaction or Hibernate `Session` if one is active. If this were not the case, then retrieving a connection from the `dataSource` would be a new connection, and you wouldn’t be able to see changes that haven’t been committed yet (assuming you have a sensible transaction isolation setting, e.g. `READ_COMMITTED` or better). ## 4.4.4 Database Console The H2 database console is a convenient feature of H2 that provides a web-based interface to any database that you have a JDBC driver for, and it’s very useful to view the database you’re developing against. It’s especially useful when running against an in-memory database. You can access the console by navigating to http://localhost:8080/h2-console in a browser. See the Spring Boot H2 Console Documentation for more information on the options available. ## 4.4.5 Multiple Datasources By default all domain classes share a single `DataSource` and a single database, but you have the option to partition your domain classes into two or more data sources. #### Configuring Additional DataSources The default `DataSource` configuration in `grails-app/conf/application.yml` looks something like this:
    dataSource:
        pooled: true
        jmxExport: true
        driverClassName: org.h2.Driver
        username: sa
        password:

    environments:
        development:
            dataSource:
                dbCreate: create-drop
                url: jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
        test:
            dataSource:
                dbCreate: update
                url: jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
        production:
            dataSource:
                dbCreate: update
                url: jdbc:h2:prodDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
                properties:
                   jmxEnabled: true
                   initialSize: 5
This configures a single `DataSource` with the Spring bean named `dataSource`. To configure extra data sources, add a `dataSources` block (at the top level, in an environment block, or both, just like the standard `DataSource` definition) with a custom name. For example, this configuration adds a second `DataSource`, using MySQL in the development environment and Oracle in production:
    dataSource:
        pooled: true
        jmxExport: true
        driverClassName: org.h2.Driver
        username: sa
        password:

    dataSources:
        lookup:
            dialect: org.hibernate.dialect.MySQLInnoDBDialect
            driverClassName: com.mysql.jdbc.Driver
            username: lookup
            password: secret
            url: jdbc:mysql://localhost/lookup
            dbCreate: update

    environments:
        development:
            dataSource:
                dbCreate: create-drop
                url: jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
        test:
            dataSource:
                dbCreate: update
                url: jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
        production:
            dataSource:
                dbCreate: update
                url: jdbc:h2:prodDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
                properties:
                    jmxEnabled: true
                    initialSize: 5
                    ...
            dataSources:
                lookup:
                    dialect: org.hibernate.dialect.Oracle10gDialect
                    driverClassName: oracle.jdbc.driver.OracleDriver
                    username: lookup
                    password: secret
                    url: jdbc:oracle:thin:@localhost:1521:lookup
                    dbCreate: update
You can use the same or different databases as long as they’re supported by Hibernate. If you need to inject the `lookup` datasource in a Grails artefact, you can do it like this:
    DataSource dataSource_lookup
| While defining multiple data sources, one of them must be named "dataSource". This is required because Grails determines which data source is the default by determining which one is named "dataSource".
#### Configuring Domain Classes If a domain class has no `DataSource` configuration, it defaults to the standard `'dataSource'`. Set the `datasource` property in the `mapping` block to configure a non-default `DataSource`. For example, if you want to use the `ZipCode` domain to use the `'lookup'` `DataSource`, configure it like this:
    class ZipCode {

       String code

       static mapping = {
          datasource 'lookup'
       }
    }
A domain class can also use two or more data sources. Use the `datasources` property with a list of names to configure more than one, for example:
    class ZipCode {

       String code

       static mapping = {
          datasources(['lookup', 'auditing'])
       }
    }
If a domain class uses the default `DataSource` and one or more others, use the special name `'DEFAULT'` to indicate the default `DataSource`:
    class ZipCode {

       String code

       static mapping = {
          datasources(['lookup', 'DEFAULT'])
       }
    }
If a domain class uses all configured data sources, use the special value `'ALL'`:
    class ZipCode {

       String code

       static mapping = {
          datasource 'ALL'
       }
    }
#### Namespaces and GORM Methods If a domain class uses more than one `DataSource` then you can use the namespace implied by each `DataSource` name to make GORM calls for a particular `DataSource`. For example, consider this class which uses two data sources:
    class ZipCode {

       String code

       static mapping = {
          datasources(['lookup', 'auditing'])
       }
    }
The first `DataSource` specified is the default when not using an explicit namespace, so in this case we default to `'lookup'`. But you can call GORM methods on the 'auditing' `DataSource` with the `DataSource` name, for example:
    def zipCode = ZipCode.auditing.get(42)
    ...
    zipCode.auditing.save()
As you can see, you add the `DataSource` to the method call in both the static case and the instance case. #### Hibernate Mapped Domain Classes You can also partition annotated Java classes into separate datasources. Classes using the default datasource are registered in `grails-app/conf/hibernate.cfg.xml`. To specify that an annotated class uses a non-default datasource, create a `hibernate.cfg.xml` file for that datasource with the file name prefixed with the datasource name. For example if the `Book` class is in the default datasource, you would register that in `grails-app/conf/hibernate.cfg.xml`:
    <?xml version='1.0' encoding='UTF-8'?>
    <!DOCTYPE hibernate-configuration PUBLIC
              '-//Hibernate/Hibernate Configuration DTD 3.0//EN'
              'http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd'>
    <hibernate-configuration>
       <session-factory>
          <mapping class='org.example.Book'/>
       </session-factory>
    </hibernate-configuration>
and if the `Library` class is in the "ds2" datasource, you would register that in `grails-app/conf/ds2_hibernate.cfg.xml`:
    <?xml version='1.0' encoding='UTF-8'?>
    <!DOCTYPE hibernate-configuration PUBLIC
              '-//Hibernate/Hibernate Configuration DTD 3.0//EN'
              'http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd'>
    <hibernate-configuration>
       <session-factory>
          <mapping class='org.example.Library'/>
       </session-factory>
    </hibernate-configuration>
The process is the same for classes mapped with hbm.xml files - just list them in the appropriate hibernate.cfg.xml file. #### Services Like Domain classes, by default Services use the default `DataSource` and `PlatformTransactionManager`. To configure a Service to use a different `DataSource`, use the static `datasource` property, for example:
    class DataService {

       static datasource = 'lookup'

       void someMethod(...) {
          ...
       }
    }
A transactional service can only use a single `DataSource`, so be sure to only make changes for domain classes whose `DataSource` is the same as the Service. Note that the datasource specified in a service has no bearing on which datasources are used for domain classes; that’s determined by their declared datasources in the domain classes themselves. It’s used to declare which transaction manager to use. If you have a `Foo` domain class in `dataSource1` and a `Bar` domain class in `dataSource2`, if `WahooService` uses `dataSource1`, a service method that saves a new `Foo` and a new `Bar` will only be transactional for `Foo` since they share the same datasource. The transaction won’t affect the `Bar` instance. If you want both to be transactional you’d need to use two services and XA datasources for two-phase commit, e.g. with the Atomikos plugin. #### Transactions across multiple data sources Grails does not by default try to handle transactions that span multiple data sources. You can enable Grails to use the Best Effort 1PC pattern for handling transactions across multiple datasources. To do so you must set the `grails.transaction.chainedTransactionManagerPostProcessor.enabled` setting to `true` in `application.yml`:
    grails:
      transaction:
        chainedTransactionManagerPostProcessor:
          enabled: true
The Best Efforts 1PC pattern is fairly general but can fail in some circumstances that the developer must be aware of. This is a non-XA pattern that involves a synchronized single-phase commit of a number of resources. Because the 2PC is not used, it can never be as safe as an XA transaction, but is often good enough if the participants are aware of the compromises. The basic idea is to delay the commit of all resources as late as possible in a transaction so that the only thing that can go wrong is an infrastructure failure (not a business-processing error). Systems that rely on Best Efforts 1PC reason that infrastructure failures are rare enough that they can afford to take the risk in return for higher throughput. If business-processing services are also designed to be idempotent, then little can go wrong in practice. The BE1PC implementation was added in Grails 2.3.6. . Before this change additional datasources didn’t take part in transactions initiated in Grails. The transactions in additional datasources were basically in auto commit mode. In some cases this might be the wanted behavior. One reason might be performance: on the start of each new transaction, the BE1PC transaction manager creates a new transaction to each datasource. It’s possible to leave an additional datasource out of the BE1PC transaction manager by setting `transactional = false` in the respective configuration block of the additional dataSource. Datasources with `readOnly = true` will also be left out of the chained transaction manager (since 2.3.7). By default, the BE1PC implementation will add all beans implementing the Spring `PlatformTransactionManager` interface to the chained BE1PC transaction manager. For example, a possible `JMSTransactionManager` bean in the Grails application context would be added to the Grails BE1PC transaction manager’s chain of transaction managers. You can exclude transaction manager beans from the BE1PC implementation with this configuration option:
    grails:
      transaction:
        chainedTransactionManagerPostProcessor:
          enabled: true
          blacklistPattern: '.*'
The exclude matching is done on the name of the transaction manager bean. The transaction managers of datasources with `transactional = false` or `readOnly = true` will be skipped and using this configuration option is not required in that case. #### XA and Two-phase Commit When the Best Efforts 1PC pattern isn’t suitable for handling transactions across multiple transactional resources (not only datasources), there are several options available for adding XA/2PC support to Grails applications. The Spring transactions documentation contains information about integrating the JTA/XA transaction manager of different application servers. In this case, you can configure a bean with the name `transactionManager` manually in `resources.groovy` or `resources.xml` file. | Quick Reference (hide)