Just adding this to the community incase anyone is having the same problem I encountered.
Environment:
Windows Server 2008 R2 SP1 - as a VM guest server on VMWare ESXi 5
4GB vRAM
40GB vHDD (single drive shown in Windows)
Altiris Deployment Solution 6.9 SP6 fresh install
SQL Express 2008 R2 used on same server
Problem 1:
When I ran the installer, I selected the "Simple Install Helper" and installed the MSDE database to the same server. I received an error message saying this database server is not compatible with this operating system. I clicked on Run anyway and I restarted the server when it asked
I then ran the Simple Install (with the tick for Include PXE server) and the installer could not see the database server.
Why does the installer want to install an old Database sever (MSDE) and not an SQL Express server (like v7 does)?
So I decided to remove this and manually install SQL Express 2008 R2, and then re-run the installer this time selecting Custom install - see problem 2:
Problem 2:
With having SQL Express R2 installed manually (accepting the defaults during the installer), resulted in an error (see attachment)
I did some searching in the forums and found this post - http://www.symantec.com/business/support/index?page=content&id=TECH213099&actp=search&viewlocale=en_US&searchid=1398700011747
I looked at the services on my server and noticed the "Altiris Deployment Server DB Management" service was not running, so I started it, but had the same error message again.
I couldnt find anymore postings that had the same error message.
In the end I changed the database server name from [name]\sqlexpress to localhost\sqlexpress in the installer settings and it connected
I can now open the Deployment Server Console
Hope this helps anyone with the same problem