How To: Generate SQL scripts from Liquibase changesets

There is a useful feature in Liquibase called Offline Database Support that allows you to generate the SQL scripts from the changesets without actually updating the database. This Offline Database Support functionality is called updateSql. This becomes handy when you cannot directly run Liquibase changesets on the target database or if the output of the changesets needs to be reviewed.

Note that updateSql checks the transactions within the DATABASECHANGELOG table to determine what SQL statements to generate. With that said, in the event that you cannot directly run the changesets against the target database, you can ask for the DATABASECHANGELOG table and use that to keep the history correct.

To streamline the process of generating SQL scripts, it would be useful to create a batch file that would contain the command and parameters. Let’s say that the filename is GenerateSQL.bat which currently contains the following values targeting an Oracle database:

C:\Dev\liquibase-3.3.2-bin\liquibase ^
--classpath="C:\Dev\liquibase-3.3.2-bin\lib\ojdbc7.jar" ^
--driver="oracle.jdbc.driver.OracleDriver" ^
--url=jdbc:"oracle:thin:@ORA-DBDEV:1521:DEVORA11G" ^
--username=DEV_LIQUIBASE_TEST ^
--password=asd123 ^
--changeLogFile="C:\Dev\LiquiBase\ORACLE\%1" ^
--logLevel=debug ^
--logFile="C:\Dev\LiquiBase\ORACLE\output.oracle.log" ^
updateSQL > C:\Dev\LiquiBase\ORACLE\output.oracle.%1.sql

where…
Line 01: Location of Liquibase binary
Line 02: Location of JDBC driver. Valid values are:

  • ojdbc7.jar (ORACLE)
  • sqljdbc41.jar (MSSQL)

Line 03: Name of the JDBC driver. Valid values are:

  • oracle.jdbc.driver.OracleDriver (ORACLE)
  • com.microsoft.sqlserver.jdbc.SQLServerDriver (MSSQL)

Line 04: Details of the database server. Valid values are:

  • jdbc:”oracle:thin:@<SERVER_NAME>:<PORT>:<SERVICE_ID>” (ORACLE)
    • jdbc:”oracle:thin:@SERVER01:1521:DEVORA11G”
  • jdbc:”sqlserver://<SERVER_NAME><SERVER_INSTANCE>;databaseName=<DB_NAME>” (MSSQL)
    • jdbc:”sqlserver://SERVER01SQL2014;databaseName=DEVSQL”

Line 05: Database server username
Line 06: Database server password
Line 07: Location of the Liquibase change set file (*.xml)
Line 09: The log file for the operation
Line 10: The file where the generated SQL scripts will be stored.

Lines 07 and 10 have “%1” which represents an argument passed during execution of the batch file. This batch file can be used as follows:

C:\Dev\Liquibase\GenerateSQL.bat ChangeSetFileName.xml

This command will create output.oracle.log and output.oracle.ChangeSetFileName.xml.sql

You can integrate this process in Visual Studio by following this article. Note that you need to change the contents of the batch file to use updateSql.

How To: Install Liquibase On Your Local Machine

STEP 1

Download all pre-requisites and dependencies:

* – Download sqljdbc_4.1.5605.100_enu.tar.gz then extract sqljdbc41.jar
** – Download ojdbc7.jar

STEP 2

  • Install Java JDK
  • Extract Liquibase to a folder on your local machine (e.g. c:devliquibase-3.3.3-bin)
  • Copy sqljdbc41.jar and ojdbc7.jar to the lib folder of Liquibase (e.g. c:devliquibase-3.3.3-binlib)
  • Modify the liquibase shell file located on the root folder (e.g. c:devliquibase-3.3.3-binliquibase.bat) by adding “-Xmx1024m” after JAVA_OPTS=
IF NOT DEFINED JAVA_OPTS set JAVA_OPTS="-Xmx1024m"

java -cp "%CP%" %JAVA_OPTS% liquibase.integration.commandline.Main %CMD_LINE_ARGS%

OPTIONAL:

  • Add Liquibase to your PATH environment variable by:
  • Right-clicking Computer > Properties > Advanced System Settings > Advanced tab
  • Click Environment Variables button
  • Edit PATH variable and put in the Liquibase folder (e.g. c:\dev\liquibase-3.3.3-bin)

Congratulations! You are now a proud owner of a computer with Liquibase.

How To: Execute Liquibase Database Change Log file (a.k.a. "driver file") from Visual Studio

The Liquibase Database Change Log file or as we refer to it, the “driver” file, is the root of all changesets. This is the file that is passed to Liquibase during execution as the changeLogFile parameter which lists all changesets that needs to be executed in order. And because we love Visual Studio, we’d like to execute the driver file right from the IDE.

First step is to create a batch file with the following contents. Place this file in a folder that you can easily remember (c:\Dev\Liquibase\LBUpdateSQL.bat):

C:\Dev\liquibase-3.3.3-bin\liquibase ^
--classpath="C:\Dev\liquibase-3.3.3-bin\lib\sqljdbc41.jar" ^
--driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" ^
--url=jdbc:"sqlserver://127.0.0.1;databaseName=TARGET_DB" ^
--defaultSchemaName=dbo ^
--username=******** ^
--password=******** ^
--changeLogFile=%1 ^
--logLevel=info ^
--logFile="C:\Dev\LiquiBase\logs\output.log" ^
update

Don’t forget to change the values for TARGET_DB, username and password. Also make sure that the paths are valid and appropriate (lines 1, 2 and 10).

Next step is to open Visual Studio and click on TOOLS > External Tools… and click the Add button. Fill-out all the fields as in below and make sure Close on exit is not checked:

liquibase-external-tool

Now this part is important. Make sure that the Initial directory points to the location of the driver file (in this case its C:\Dev\git-repos\DatabaseScripts\(project_name)\updates). This will ensure that the filename field in the DATABASECHANGELOG table only contains the filename of the changeset and not the full path. To understand why this is important, read more here.

Click on Ok to close the External Tools window.

Lastly,open the driver file in Visual Studio and click on TOOLS > Liquibase (this is what you placed in the Title field in the screenshot above).

run-driver-run

There should be a command window that pops up that shows if the execution has failed or succeeded. The equivalent of all of this is like executing the batch file via command window while on the folder that contains the driver file with the driver file as an argument.

How Liquibase Considers A Changeset As Unique

We have a sandbox here in the office that has a database that we run the Liquibase Database Change Log file (a.k.a. “driver file”) against every now and then. We do not develop against this database since one of its purpose is to ensure that the latest version of the driver file runs without problems.

A few days ago, I took a backup of that database and restored it to my local machine. According to the DATABASECHANGELOG table, the driver file was last ran on 17 JUN (see screenshot below). However, when I tried running the latest Liquibase driver file from /develop against my now local db, I got a variety of errors ranging from duplicate key values to tables/columns already existing.

databasechangelog-table

I might have missed the email but last time I checked, the idea was to allow running and re-running of the changesets without having a negative effect and without having the execution error out. I had to exclude a bunch of changeset entries in the driver file to get it to finish without reporting any errors. Below is the list with the reason why it failed:

liquibase-problem-files

An officemate tried to run the latest driver file on his local machine and worked without any problems. We checked the DATABASECHANGELOG table for the existence of the IDs of the changesets and to our surprise, they were already there. The errors being thrown due to the lack of precondition in the changesets are just a manifestation of a different problem altogether (although the lack of precondition on the changeset is a problem on its own). We were able to come to that conclusion because Liquibase will only try and run the changeset if that changeset’s ID is not on the DATABASECHANGELOG table. So why is Liquibase trying to execute a changeset when the ID of the changeset is already in the DATABASECHANGELOG table?

Upon further investigation, we came upon this question on StackOverflow: Liquibase tried to apply all changeset, even if database is present‌
The take away is that Liquibase tracks each changeset as a row in the DATABASECHANGELOG table which is composed of the combination of the “id”, “author” and a “filename” column.

If we go back to the first screenshot above, you’ll notice that the value for the FILENAME column after 17 JUN 2015 has changed. At this point, it means that I’ve restored a backup of that database to my local machine and the path where I’ve been executing Liquibase is different from the path where Liquibase was being executed on the sandbox in the office.

So how did we solve the problem?

1.) We cleaned up the filename column so that it only contains the actual filename of the changeset.*

2.) Made modifications on the way we were executing the driver file moving forward so that the FILENAME column only contains that – the filename without paths. (How To: Execute Liquibase Database Change Log file (a.k.a. “driver file”) from Visual Studio).

fixed-filename-only

.* truncated the DATABASECHANGELOG table, dropped all affected tables and ran the driver file.

CKEditor.Net Does Not Work Inside Multiview

I’ve written in a post a few months ago about a javascript-based lightweight WYSIWYG rich-text editor that can be used in web pages called FCKEditor. The developers have upgraded it and called it CKEditor. I tried to use it two days ago and up until now, I still cannot use it for my specific purpose. I am near the end of a project with this WYSIWYG editor as the second-to-the-last-module. I am convinced that I have encountered a bug and have yet to find a solution. I will blog about this for documentation purposes.

CKEditor + AJAX + UpdatePanel + Multiview = Problem

The bug I am encountering now appears only on when the conditions are right. I created a new AJAX-enabled website project in Visual Studio 2008 with C# as my code-behind. I downloaded the latest (as of time of writing) version of CKEditor 3.5.2 and CKEditor.Net 3.5.2 control from their website.

From Solution Explorer in VS2008, I added a BIN folder and placed the `CKEditor.NET.dll` (from CKEditor.Net 3.5.2) there. The ckeditor folder that contains the core files (from CKEditor 3.5.2) was also placed in the root of the website. I opened the Web.Config file and added the following line to the node:

<add tagPrefix="CKEditor" namespace="CKEditor.NET" assembly="CKEditor.NET" />

Then, I opened Default.aspx and add the following line:

<CKEditor:CKEditorControl runat="server" ID="tbContent1" />

So that the whole Default.aspx would look like this:

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="Default.aspx.cs" Inherits="_Default" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title>Untitled Page</title>
</head>
<body>
    <form id="form1" runat="server">
    <asp:ScriptManager ID="ScriptManager1" runat="server" />
    
</form> </body> </html>

If you try to view it by debugging it (hit CTRL+F5 while on VS2008), you should see the CKEditor.Net Control working as intended.

Now if we put the CKEditor.Net control inside an update panel, it would still work. If the update panel contains a multiview control with only one view panel and the CKEditor.Net control is inside the only view panel, it would still also work. But once the multiview control has two or more view panels and the CKEditor.Net control is on the second view panel (or any panel for that matter, as long as it is not on the default active view index of the multiview) that’s where things start to get ugly.

We can change the code of Default.aspx to:

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="Default.aspx.cs" Inherits="_Default" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title>Untitled Page</title>
</head>
<body>
    <form id="form1" runat="server">
    <asp:ScriptManager ID="ScriptManager1" runat="server" />
    
</form> </body> </html>

Notice the button on the first view panel. We’ll be using that to switch the active view index of the multiview control to the second view panel. We need to handle the OnClick event of that button so we do the whole code-behind (Default.aspx.cs) file like so:

using System;
using System.Data;
using System.Configuration;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {

    }

    protected void btn_Click(object sender, EventArgs e)
    {
        //MultiView1.ActiveViewIndex = 1;
    }
}

Once we debug it again (CTRL+F5 while on VS2008), we get a button on the initial page load since the multiview’s active view index property is set to zero (ActiveViewIndex=”0″). If we click on the button, the OnClick event of the button sets the multiview’s active view index to 1 (where the CKEditor.Net contol is at). This produces a javascript error and the CKEditor.Net control is rendered as a plain textbox.

 

Line: 5
Error: Sys.ScriptLoadFailedException: The script '/CKEditorMultiview/ckeditor/ckeditor.js?t=B1GG4Z6' could not be loaded.

 

It happens on Internet Explorer 8.0.7601.17514, Google Chrome 10.0.648.151, FireFox 3.6.13 (although the javascript error does not show on GC and FF – might be a in-browser error config thing – the CKEditor.Net being rendered as a plain textbox is present on all three major browsers).

As I’ve said at the start of this post, I have not yet found a solution to this problem. If you have any information on how to fix this particular problem, kindly drop me a line at the comments. Thanks!

Using FCKeditor with Asp.Net

FCKeditor (now known as CKEditor) is a javascript-based lightweight WYSIWYG (What You See Is What You Get) rich-text editor that can be used in web pages. It is an open-source project created by Frederico Caldeira Knabben way back in 2003. It is designed to make the text being edited to look more-or-less the same with how it will look like when published. A good example of a WYSIWYG text editor is what you get when you compose a new email message in GMail or when you try to create a new blog post in WordPressBlogEngine. 😀

It enables the web user to experience common text editing features (that were found only on desktops once upon a time) such as changing the font colors and highlights, find and replace, spell check, set text/paragraph justification, basic styling like bold, italics and underline. 😉

Before we get started, we need to download the necessary files:

  1. Main code – actual codes for the FCKeditor
  2. Assembly for .Net – Asp.Net control for easier integration

The FCKeditor is hosted at Sourceforge.

Click on the link and download the latest version of the FCKeditor main code (fig. 1) and FCKeditor.Net control (fig. 2.). Each has their own sub folder so you may have to browse through them to get to the actual .zip file. For this example, the versions that we got are 2.6.6 for the main code and 2.6.3 for the ASP.NET control.

fig. 1. FCKeditor main code

fig. 2. FCKeditor Control for .NET

Extract/uncompress these .zip files and remember where the uncompressed files are located. For this example, let’s put them in c:FCK so that the main code will be at c:FCKFCKeditor_2.6.6 and the FCKeditor Control for .NET will be at c:FCKFCKeditor.Net_2.6.3

Integrating it with a new project

  • Create a new web site project in Visual Studio 2008. We’ll call the sample project FCKeditorSample. (Creative names as always  :)) )
  • Right-click on the web site project and add the ASP.NET folder “Bin” (fig. 3.).

fig. 3. Adding the Bin folder

  • Right-click on the Bin folder and click “Add Existing Item” then browse to where you extracted the FCKeditor Control for .NET (c:FCKFCKeditor.Net_2.6.3binRelease2.0). Select the FredCK.FCKeditorV2.dll assembly file and click the Add button (fig. 4).

fig. 4. FredCK.FCKeditorV2.dll assembly file added to Bin folder

  • Go to where you extracted the FCKeditor main code (c:FCKFCKeditor_2.6.6) then right-click and copy the “fckeditor” folder. Go back to Visual Studio and right-click on the web site project and click paste (your Solution Explorer should look something like in fig. 5.).

fig. 5. Files of the main code added (click to enlarge)

  • Before we can use the FCKeditor control, we must register it in our application. We have two options to do this: a.) per page via register directive (see listing 1) or b.) per application via Web.Config file (see listing 2). 
<%@ Register TagPrefix="FredCK.FCKeditorV2" Assembly="FredCK.FCKeditorV2" tagPrefix="FCKeditorV2" %>

listing 1. Register directive per page

 

If we are to use the per page method of registering the control, we must do so for every page the will use the FCKeditor in our application. While this works fine, maintenance becomes a problem especially if we have a lot of custom user controls. x_x

<pages>
 <controls>
  <add tagPrefix="asp" namespace="System.Web.UI" assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
  <add tagPrefix="asp" namespace="System.Web.UI.WebControls" assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>
<add namespace="FredCK.FCKeditorV2" assembly="FredCK.FCKeditorV2" tagPrefix="FCKeditorV2"/>
 </controls>
</pages>

listing 2. Registering the  control in the Web.Config file

 

If we use the second option, open the Web.Config file and locate the “controls” node under configuration <system.web> pages then create the “add” node similar to line 5 in listing 2. I prefer to use this method because you only need to do it once and every page in our application can use the FCKeditor. 🙂 or any other control for that matter.

  • After registering, we can now use the FCKeditor control like any other ASP.NET control. Open Default.aspx and modify it such that the form tag looks like this:
<form id="form1" runat="server">
 
</form>

listing 3. Using the FCKeditor control

 

We also need to specify the BasePath so that we know where the main code of the FCKeditor is located. The ~ (tilde) means that we resolve the path starting from the web application’s root path. It is a much easier way of handling paths instead of “..” all over our web application. 😉

  • Hit Run and after the web browser has launched, you’ll be greeted with the default FCKeditor tool set (fig. 6). 😀

fig. 6.Using the FCKeditor in ASP.NET

8. You can access the user’s input text with formatting by using the Value property of the control (listing 4).

lbDisplayLabel.Text = FCKeditor1.Value;

listing 4. Accessing the user’s formatted input

 

I hope this helps!

Using Asp.Net Content Pages

My last post which was Asp.Net-related was about the use of master pages. Today, I am writing about Content Pages and how to use it together with Master Pages.

Having re-read the first Asp.Net-related post, I realized that it is better to include a step by step instruction via screenshots while doing all of these inside Visual Studio. In my next articles, I will definitely include screenshots but for now, please bear with me as this is my first blog for coding and technology-related stuff.  x_x

It is actually very straight forward and will be much, much simpler to understand. When creating a new content page in Visual Studio, you will be given an option to choose which master page you’ll use (yes, multiple master pages are supported). After which, the following lines of code will be the pre-populated for you.

Home.aspx

<%@ Page Title="" Language="C#" MasterPageFile="~/MasterPage.master" AutoEventWireup="true" CodeFile="Home.aspx.cs" Inherits="Home" %>
<asp:Content ID="Content1" ContentPlaceHolderID="head" Runat="Server">
<!-- Commented text -->
</asp:Content>
<asp:Content ID="Content2" ContentPlaceHolderID="ContentPlaceHolder1" Runat="Server">
Test content
</asp:Content>

The usual HTML tags are nowhere to be found as it will be “inherited” from the master page specified. We can just focus on the functionalities of the page that we are working on instead of worrying about the uniformity of the user interface for the whole website.

In line 1, notice that the MasterPageFile attribute is set to the name of the master page that we created in the previous article.

Lines 2-4 and 5-7 are the content areas where we will be placing our page content. These can be images, textboxes or any user interface elements. If we look back at our previous article, these corresponds to the content place holders located in the master page’s head and body tags, respectively.

If we are to display this page together with the masterpage, the code would – more or less – look like this:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head id="Head1" runat="server">
    <title></title>
    <!-- Commented text -->
</head>
<body>
    <form name="aspnetForm" method="post" action="Home.aspx" id="aspnetForm">
    <asp:ScriptManager ID="ScriptManager1" runat="server" />
    
Test content
</form> </body> </html>

On my next article, I will be including screenshots with Visual Studio in action and more examples of Asp.Net Master Pages and Content Pages.  😀