Learning by doing

Recently at work, we have been discussing the question of Perl versus Python for file copy, ftp, email and other daily, weekly or monthly automation / integration tasks. We considered Perl because some existing tasks had already been written in it, and Perl has a reputation as the System Admin's language of choice. I took a task that needed automated and wrote it in Perl 5.8, then refined it using CPAN (sweet CPAN) for ArgvFile, Log4Perl, Carp, and Date::Calc. I find Perl to be very functional and completely able to meet my needs, but very quirky or perhaps unsettling. You see, I have been an object oriented programmer for a while now, and Perl's class system just feels "bolted on" to me. And I really had to jump through hoops to do a try/catch, but using eval with a return flag and an END block to test the return worked acceptably. I didn't test the exception module, as I did not discover it in time. Next I tried Python 2.5. After a quick brush-up on the...

NHibernate 2.0 is not even close to release, but it is shaping up nicely. Oren Eini (His blog is called Ayende @ Rahien) has a list of interesting new features . Dario Quintana has something to say as well . He has contributed some code to the uNHAddins for Active Record Detached Query support . The Castle development team is considering pointing to NHibernate Trunk (which will be released as 2.0 eventually) so they/we can take advantage of the new features. If Castle Active Record uses NHibernate trunk, I would advise you to be diligent wit! your test code, and don't forget to run the Castle test code too. You can track Castle Project changes on FishEye .

I follow the Mono Blog aggregator , and Miguel de Icaza mentioned that a user had tested ProMesh.NET on Mono , and it ran without any changes! That in in itself was cool, but when I took a look at ProMesh.Net , it looked like a well designed framework, perhaps comparable to MonoRail . The framework is ORM agnostic, and the biggest weakness might be it's view templates, but they have a great tutorial , so judge for yourself!

I am a registered user on SQL Server Central , and I read a good article there on " Enums in SQL Server ". I thought that I would quickly present my own humble solution to this problem, although I do not mean to imply that my way is better - you can decide for yourself. First, the problem to be solved is to represent a limited set of choices, where new choices would typically require a developer's intervention to implement the business logic. An example would be a list of task priorities in a to-do list. CREATE TABLE Priorities( ID tinyint not null identity(1,1) PRIMARY KEY, Code char(3) not null unique, Priority varchar(20) not null ) ; INSERT INTO Priorities (Code, Priority) VALUES ('911', 'Emergency') ; INSERT INTO Priorities (Code, Priority) VALUES ('HIG', 'High') ; INSERT INTO Priorities (Code, Priority) VALUES ('NOR', 'Normal') ; INSERT INTO Priorities (Code, Priority) VALUES ('LOW', 'Low') ; IN...

I do not believe that I have covered this before, but the Active Record Code Generator can also generate audit records for your database tables. The template that I wrote ( SqlAuditTrigger.vm ) adds records to an AuditChanges table, with a related AuditFields table containing the table and field name. I think that this is a fairly compact solution, but I also added code to the trigger to ignore SA (system administrator) changes. This may not be appropriate for your application, but it is easily removed. I also have a "TODO" item to detect related lookup tables (really ENUMs) and translate IDs to Names, so that changes to the FavoriteColor_ID field would look at the FavoriteColor table, and instead of recording FavoriteColor_ID changed from 2 to 5, it would say changed from "Ruby Red" to "Passionate Purple". The reason that I did not actually implement it is that I do not have enough information. My code generation schema does not list all of the f...

Today, I needed to send some private data to another company, and I could not get FTP with SSL to authenticate over our firewall, so I had to sent an encrypted file over FTP. The other company has a GPG public key, so I looked around for a bit on the web (yes, I googled c# gpg encrypt open source) and found a message that mentioned Bouncy Castle on the 4th or 5th page (why was it so far down the list???). I then googled Bouncy Castle to finally find that they have a wonderful c# implementation of GPG encryption (and many others), and their test code includes an example that reads a file and writes an encrypted file. This made integration VERY simple. I did move their test folder into their test project, so I would not be deploying all of the test code to production.

I was reading a blog on MSDN, with a link to some cool Power Toys for Visual Studio 2005 Team Edition for Database Professionals. So today, I had some time to install the VS2005 TE Service Release 1 so I could grab the power toys, but the microsoft link is dead! The developer went on a vacation (literally), so it might be a while until this gets fixed. Oh, by the way, O’Reilly Hacking Visual Studio mentions a cool documentation tool called GhostDoc . It looks slick! Update: Here is the link , thanks to Michael!

Friday, I added an NHibernate mapping.hbm.xml template. I mainly wanted to show that the code generator can be used for other purposes than just Castle ActiveRecord, and NHibernate is an easy target. Jim R. has requested that I add HasMany (aka ManyToOne) mapping properties and attributes. I think this is going to push me to refactor the schema extraction code. Some of my future goals for the application are: 1. Full Model Generation (list of children, whether directly or indirectly through a many-to-many table) 2. User controllable plural-to-singular mapping. 3. Multi-database, using the Database Provider Factory. 4. Refactor code generation into a library (as I do with all projects) 5. Command-line code generation (supply at least a template and a list of tables) 6. Templates for other ORM tools (pure NHibernate, Rocky Lhotka's CSLA Business Objects, etc.)

I have improved the Inequality test in the model template, as requested by JimR . This was another case where it worked well for me, only because I followed the convention that primary keys are named ID, and foreign keys are named SingularOfTable_ID. I know that I should really push the Inequality test into the template, but for now, I have corrected it in the DbFieldInfo class.

I just updated my code generator to optionally generate validation attributes. This simple change includes App.config file entries for all check boxes, and a new checkbox for "Validation" - aka validation attibute generation. While I was making this change, I realized that I really need to pass a CodeGenerationContext object to the DbTable, DbField and ModelGenerator classes. The requester can populate the context, and pass it to the code generator. Anyway, enough about the code, let's talk about the templates. I made a simple template this weekend to generate a DataGridView column array, suitable for databinding. I'm sure my new template will need some tweaks to handle Foreign Keys better (it currently just displays them as TextBox). Let's look at a template. ##FILENAME:PR_${table.GetClassName()}_Insert.sql ## ## Generate a SQL stored procedure to insert a record into the ## specified table and return the newly created primary key ## ## FUTURE: The generat...

Thanks to some recent interest in the community, I have made a few more changes to ActiveRecord Code Generator . First, I added a browse button and a dialog, using some code supplied by Jim R . I don't care for the folder browser dialog, but it is built in to .Net 2, so I use it. Second, I fixed a few bugs that crop up when a table does not match my arbitrary table and field naming scheme. This project was written quickly as a pre-requisite to a large project, and it shows, but it is steadily improving. Evening Update: I added a POCO (Plain Old C# Object) template as a starting point for an NHibernate cs and xbm template set.

Today, I updated the ActiveRecord Code Generator a bit. I checked in changes to use primary and foreign key details from INFORMATION_SCHEMA. The original code used naming conventions to decide what various fields were used for - ID = Primary Key, Field_ID = Foreign Key to table Fields. If you want to use naming conventions, let me know and I can add a setting in App.Config to allow this (along with any "real" key constraints).

Today, I was inspired to hack the context menu in Firefox 2.0, removing "Send To" and "Send Image". I googled for a solution, and found step-by-step directions for Firefox 1.0 . So, I started digging, and the over-all instructions were good, but I only needed to comment out the menu options in browser.xul, and not touch the javascript in browser.js. The original instructions were a bit vague, but in the end, it worked!

I have finally released my Code Generator to Google Code as Active-Record-Gen . What does it generate? It generates ActiveRecord classes mainly, but I have used it to generate stored procedures and sys-admin scripts as well. This code generator does not (yet) generate a full Windows application project or a Mono-Rail web site, but the generated code could be used in either. In fact, with a few tweaks, this could be used to generate NHibernate "poco" and .xbm files. If you want to know more, look at the screen shots above, or head over to Google Code and run it. In my haste to make my first EXE release before supper, I forgot to add the Template directory, which should be at the same directory level as the EXE and config files. I just (1.5 hours later) uploaded a new EXE, but 2 people have already downloaded the EXE (not the source though). As for the basic table object, it is built with the following assumptions: Table name is plural, class name is singular. Field ...

Update 05/22/2007: No response from the community. I think Hammett is working on releasing a new version of Castle (It's much more than just ActiveRecord!), so I have not received any feedback on my patch. I have just posted patch AR-156 to Castle's ActiveRecord project to add support for XML free mapping of insert, update and delete stored procedures to ActiveRecord classes. This requires a bit of work, since the stored procedures must match the field order generated by NHibernate , rather than being controlled by the application developer. This means that we have to create the class normally, run it through a test cycle with logging turned on, capture the prepared statement's field order, and then re-write the stored procedure to match what NHibernate wants. This is because behind the scenes the stored procedure replaces a prepared sql statement that NHibernate would otherwise be using. This will be a mess to maintain until or unless someone contributes a patch to ...

Update: the latest versions of Reflector and Doubler are working well together. I just noticed that Jay Flowers has released a version of the Doubler Add-In that works with the current Reflector by Lutz Roeder . I tried installing Doubler in a sub-directory under Reflector, but it failed to load, so I had to move it into the Reflector directory. The current version of Doubler generates test stubs with a mis-spelled attribute (Memeber instead of Member), but it will generate test stubs, interfaces, recording wrappers, mock objects, etc. It seems like it will be quite handy, with support for NUnit, MBUnit, etc. although Doubler forgets all of your settings every time you move to a new class. That is almost a deal breaker there, but at least you can select an assembly and generate all tests at once. I have to change the output language to c#, change the output path (or just build where the tool wants to go, and copy it), change the default test framework, and set "WithTestAtt...

NHibernate was released May 3, 2007 and they have a migration guide for those of you who do not follow changes in the svn. This release supports stored procedures, and as you can see in the screen shot above, I am working on adding SQL stored procedure generation to my ActiveRecord Generator. I have templates for INSERT, UPDATE and DELETE now, and I will write a SELECT by primary key today. It's too bad I can't just write an attribute to append to the NHibernate mapping file that ActiveRecord creates. I don't know if I will have to write my own ActiveRecord attributes, but the Castle Project seems very friendly to patches. This is very important because I would not really want to release a program that requires a patched library. I have found several bloggers who have created NHibernate + Stored Procedure samples, but none that have ActiveRecord samples (since we do not have the attributes yet). If you have been working on something, now is your chance to share. (...

If you've been reading my blog, you know that I am developing an application using Visual Studio 2005 and Castle's ActiveRecord framework. Recently, I bought Visual Studio 2005 Pro so I could use Crystal Reports. Believe it or not, The user was able to install my application without a working version of .Net 2.0 around. As far as I can tell, they did a Windows Update, and somehow it corrupted their video drivers. Then they did a system restore back to before .Net 2.0 was installed. After this, they installed my application, and it crashed (of course!). They tried to repair .Net 2.o Framework, but that didn't quite do it. So now I will probably be connecting via Remote Desktop to survey the damages. I learned something from this, and I will probably have two tiny executables in my bin folder ... a "hello world" application that maybe says "If you can see this, then .Net 2.0 is installed." and maybe does some sniffing around to see if an important...

I know this entry is off-topic, but yesterday was pretty amazing. A co-worker told me about , and I signed up last month. Blingo is a search engine that uses Google's database for results, but as you search, you have a chance to win something. Blingo has "Thousand Dollar Thursdays" and sometimes they even give away an automobile! So yesterday I was trying to figure out why my ABit A8N SLI Fatality motherboard was crashing Windows XP since I added a SATA hard drive and a DVD burner. While I was searching, I won a Fandango movie ticket! That was cool, but I kept searching, and found that I needed to turn off the "[x] Let BIOS decide" option, and then turn off Command Queuing. So 30 minutes later, I did a different search and won again! That is 2 wins in a row! On the side panel, it lists who won what, and it showed me twice. The co-worker who referred me was ecstatic, as he has never won anything on Blingo so far. I use Firefox, and I like a clean...

[Update 1: finished the entry, and added some compilable code.] [Update 2: Updated the compilable code - improved the Property setter code.] Recently, I have been working on a project with Castle's ActiveRecord support. Since I am changing my database schema as I discover new requirements, and adapt to new data sources, I occasionally run into query / model / table mismatches. Now, gross mismatches cause NHibernate to choke, and refuse to accept my models, but if I have merely added a field, that will be ignored until insert time. Even then, it might not cause an exception if it is a nullable field. Now when I first started on this project, I wrote a simple SQL procedure that selected a list of fields from the database schema, walked through a cursor for each field and PRINTed code, then I would create a class file in Visual Studio 2005 and paste that code in. -- Get a list of tables SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME != 'sysdiagrams...

Google has donated Hibernate Shards - a Java 5 add-on to Hibernate that allows application driven data partitioning with your custom or their pre-build configurations. One thing that caught my eye was that when they covered partitioned Native ID generation (aka Identity), they mentioned having Database A use a range 0-200000, Database B use range 200001 - 400000, etc. I know that MySQL cluster suggests using something like this: Database A: Starting ID 1, Increment ID by 3 (number of databases) Database B: Starting ID 2, Increment ID by 3 Database C: Starting ID 3, Increment ID by 3 This allows a hands-off approach, and easily lets you divide ID by 3 and use the remainder to map to a "Shard". Of course, if your data changes significantly, you may have to dump and reload to add another database. One way to avoid a dump and reload is to pick an increment that is larger than your actual database count, and just drop an extra database in an the available Starting ID. This ...

As I have said before, I am developing with Visual Studio 2005 on .Net 2.0. This summer, I will be creating an application with integrated html help. Of course, my first inclination is to refresh my knowledge of help systems by acquiring Microsoft's HTML Help SDK . The HTML Help Workshop looks like it will do light project management, and it even has an image cropping and screen capture program called "Flash". I did not see any full downloadable examples in the MSDN Help SDK area, but they had a link to the Help Wiki . If you browse through the wiki, you will find some good (but dated) help info about C# . DotNetJunkies has a nice step-by-step c# example for integrating help after you have a compiled help file. I am thankful that my application uses .Net 2.0, as .Net 1.x did not have much support for help systems. The .Net 2.0 HelpProvider seems like it will be very useful (here is a tutorial . HTML Help Tidbit: Your help system can be thought of as a self-cont...

I have submitted a patch to add MultiQuery support to ActiveRecord. The link is at http://support.castleproject.org/browse/AR-139 . I added ActiveRecordMultiQuery.cs and updated ActiveRecordBaseQuery.cs to allow subclasses to be aggregated (added) to the MultiQuery. I did this to speed up initialization of a form (I know ... premature optimization is the root of all evil). If you like this patch, log in and vote for it! Update: The patch has been applied. In the mean time, let me caution you that ActiveRecordCriteriaQuery from the svn repository (which I also wrote and contributed) and any other "query" that uses criteria will not correctly participate in a multi-query. But if we give the NHibernate team some time, I'm sure this will be resolved. Regarding the implementation, the only thing that I really wondered about was the "root type". This basically selects the session to be created. So all queries in the multi-query need to be registered with the ...

Update: I have contributed patch AR-156 that allows full integration of Insert, Update and Delete to ActiveRecord models . If you've been reading my blog lately, you know that I have been seriously testing the Castle ActiveRecord framework out. I really love it, but I have an existing Microsoft SQL Server database with many stored procedures in it. I have tested the ActiveRecord model out, and I am sure that I will learn enough to be able to use it for standard CRUD (create, read, update, delete aka. insert, select, update, delete) functionality. BUT ... If I really want to integrate with my existing billing procedures, etc, I will have to be able to call stored procedures. I have taken two approaches ... write the ARHelper.ExecuteNonQuery(targetType, dmlString) method that gets a connection for the supplied type, executes dmlString, and closes it. write the ARHelper.RegisterCustomMapping(targetType, xmlString) method that allows me to add mappings that refer to my auto-gener...

My current development environment is Visual Studio Express C# Edition (read that as free ), Castle ActiveRecord's latest svn trunk(usually within a few days), and NHibernate svn trunk. As of NHibernate version 1.2.0, there is a very cool new class out there ... DetachedCriteria. This class lets you set all of your Castle relational attributes like BelongsTo, HasMany, etc. as lazy fetch, and over-ride this for searches, reports, or anytime you know ahead of time that you will be touching the related classes by calling detachedCriteria.SetFetchMode(..., FetchEnum.Eager). As a good netizen, I have tried to contribute to NHibernate and Castle ActiveRecord even if only in the smallest of ways . Oh yeah, I tried mapping to a SQL VIEW, and it worked GREAT! I received a comment after my last post, indicating that there is a better way, and I am sure of it, but the view guaranteed that I only have one database request for my dataset. NHibernate was wanting to re-fetch my missing as...

Lately, it feels like I've been wading through mud. As if the ActiveRecord library has been doing nothing but preventing me from getting my work done. In my dreams, I could call MyEntity.Find() with a list of ICriterion objects, and get back what I want. I was having problems with that, because I want to select several object, left joining to a notes table, and return only the most recent note (with max(NoteDate)). While I can think it, and write it in raw SQL, it has been tough doing this in an ActiveRecord query. Just to see if it is possible, I tried HQL, and finally resorted to raw SQL. I learned that "raw SQL" has several requirements imposed on it: * The returned result set must include the complete field list of all entities that you want to return (as far as I can tell). * NHibernate will replace {nt.*} with a complete field list if you include "nt" as an alias for typeof(MyNote). * Do not list an alias unless you are returning that entity in your ...

Update May 25, 2007: ActiveRecord now supports DetachedCriteria, which eliminates the need for the SlicedFindAll that I wrote below. It is nice when a library moves to add support for such commonly needed functions. So in summary, use Detached criteria instead of the code below. It is still a nice example of using NHibernate sessions. I have a history log, where each history record "belongs to" a service record. I have to treat this as a child-to-parent join, since some children are orphans. I wanted to use the FindAll(Criteria), but I wanted the option to have optional criteria, orders and aliases. My solution was to create an ARAlias class to represent an Associated Entity and an alias, and then build an ARBusinessBase class with the following method: public static T[] SlicedFindAll(int firstResult, int maxResults, Order[] orders, ARAlias[] aliases, params ICriterion[] criteria) { IList list = null; ISessionFactoryHolder holder = ActiveRecordMediator.GetSessionF...

While looking for Castle ActiveRecord examples, I found a blog by David Hayden - post 1 (design pattern overview) , post 2 (some details on a possible implementation) and post 3 (using the Castle Project). David had a series of articles about Rocky Lhotka's .NET Business Objects (one for each chapter), which I have also read and tried to apply. Don't get me wrong, Rocky's CSLA library works, but as of .Net 1.1 it required a great deal of code on my part. There are reasons why I and many others were investigating code generation tools. I have been studying Castle's ActiveRecord framework for a week and a half, and it is awesome for new development and clean data. During this discussion, remember that ActiveRecord mostly generates the XML mapping files for you and delegates the heavy lifting to NHibernate . If you have a legacy database with "sometimes there" keys, the framework will punish you severely. Below is an example of a legacy scenario that wi...

Since this is my first post on the subject, let me introduce the Castle project - a .Net framework that I have been exploring recently. It has a project that tries to mimic ActiveRecord in Ruby-On-Rails, although the Castle version of ActiveRecord uses NHibernate. The good news is that an ActiveRecord class doesn't require any XML configuration or mapping files. The somewhat bad news is that you DO have to declare your classes and populate them with properties, private members and AR mapping attributes. Since I will probably use SQLServer 2000, I will just write a few queries to extract the data type and field list for each table that I want to support. More on this later. For today, I wanted to extract the class-to-database mapping information that Castle's ActiveRecords feeds to NHibernate. I finally looked through the source code a bit and after a blind alley (NHibernate.Cfg Configuration is mostly for database connections), I ran across the ActiveRecordModel class...