Tuesday, February 17, 2009

Dashboards (CDF) with Pentaho

My first day trying to make a dashboard with Pentaho. There is a lot in this project and the documentation could certainly use some work, even from just navigating it. The IRC group ##pentaho has been extremely helpful in getting me started.
Before you start you need the Pentaho bi-suite installed and working. I'm using version 2 stable on Linux.
Then you download the CDF. That link goes to the wiki and has a link to the download. There are directions to use the installer, which is a jar file. Stop your server before you start this process.

I did the java -jar CDF.installer.jar and followed the directions.

When it asks for your solution directory, they mean pentaho-solution directory.

It says you finish and you restart the server. Then log in to localhost:8080 and you might see the CDF directory available in the browser on the left of you web browser window. I did not.

What I to do was go to the tools menu -> Refersh -> Refresh data sources.

Then everything showed up for me and I could run the examples.

Let me know if you start using CDF.

Thursday, February 12, 2009

Dual Monitors with Ubuntu 8.10 and nVidia 9600GT - Sweet

First thing, it does work. I say that because it was touch and go for a while. It took 12 hrs to get this working, so get some coffee and plan to be really persistent before you even think about it.
Second - It could go super easy, but it might not. So back up your xorg.conf before you start. I know that would stupid to say - because I didn't! I'm all in but you might want to think twice about it.
My goal was to get my Quad running 8.04 with 4GB ram running dual monitors - Viewsonic widescreens.
If you are going to try this make sure you follow the directions that fit your hardware and distro. I saw a lot stuff about compiling everything, and that was old and not needed.
I got the card, put it in and tried to boot up. I got a nice black screen, so it didn't start off well. I was on 8.04 at the time. I tried installed the nVidia drivers via ssh and followed the instructions, but it just wasn't working. After a few hours, I gave up and did an upgrade to 8.10. That took a couple of hours but went smooth.
Then I did the reboot and up came failsafe mode or 640x480. This time I could follow the normal 8.10 directions. Go to hardware drivers and select the proprietary nVidia driver 177 I think was recommended and I took that one. I restarted X but still 640x480. I rebooted :) still 640x480. :(
I ran dpkg-reconfigure xserver-xorg a few times with and without -phigh. Don't do this, that will never work. I found that it drops a vanilla xorg.conf onto your machine and doesn't specify the driver as nvidia with the a bus entry - you don't need to know this if you don't do it. If you did do it, go back to the version that was installed with the proprietary drivers.
I went into xorg.conf and deleted all the lines that set the resolution to 640x480 and restarted X (ctrl-alt-backspace). Wham it worked. It restarted with one monitor in 1680x1050. Then I had to open nvidia-settings.
Do this from the command line as sudo or you can't rewrite the xorg.conf file.
Once I did that twice, I was able to just select the second monitor to activate it and it defaulted to 1680x1050. I hit save and was a afraid to touch it anymore.
First impression is the card was great, X performed so well, I was surprise. As good as using my MacBook and a second screen.

Couple of things I still need to do:
1. Turn on Compiz and see how it goes.
2. My background was stretched across both monitors and I don't like that.
3. I think I need to fix my Xvnc, that seemed to stop working.
4. Onc LCD shows as a CRT. I think that is affecting the brightness. Not sure why that's wrong.

On the plus side, I'm not using the IBM on-board graphics card anymore and that freed up an mtrr which you only get 8 of and I kept getting warnings about being out. Then I only had 1M uncachable entry. It didn't solve my memory problems, but its an improved situation.

Looks cool too.

Saturday, February 7, 2009

Ubuntu 8.04 remote access from MacBook

I wanted to get into my Linux box from my MacBook. This solution should work from other computers as well.
On you linux box follow the directions here.
Pretty basic and I did them remotely from mac over ssh.
One thing is for the MacBook you should change the resolution 1024X768 to 1280x800 and then you can use fullscreen mode. I'm using it now to type this blog. This was done in the Xvnc file you created in the docs above.
This runs over ssh. They don't really tell you how to do that on Mac, they give the Putty Windows instructions.
Its just a simple tunnel:
ssh -L 5901:localhost:5901 username@host.com

Then I used Chicken of the VNC which I had already downloaded. You just connect to localhost. Then it will get forwarded via ssh to your server.

Gnome crashed for me on my first login. I created a new remote login and then loaded
gconf-editor
from the shell. I had to go in their to apps/gnome_settings_daemon/plugins and made the mouse and keyboard plugins inactive.
I did this on a new login, becuase i didn't want to mess with my existing. I also wanted to tweak some settings, like a basic color background and no theme. This way I could turn off any animation etc that would just slow down vnc.
This change stopped the crashing and system has been working well. I made one last change, I went in to the login window settings and under the remote tab, I set it to be the same as local.
Would I wan to do this all the time, probably not, but it is certainly functional as I have had no trouble writing this blog over a remote vnc session.

Good luck

Fixing xvidcap crashes on Ubuntu

Using xvidcap from the Ubuntu repository doesn't seem to work too well, at least for me. The version in the package tree is 1.1.6. That version would never achieve the frame rates I wanted but even worse it crashes all the time. It was crashing within 5 secs some times. Hardly enough time to make any type of video. I think I have it working on my machine now and I thought I would share what I did.
I did search the repository for updates, but for 8.04 1.1.6 is the latest. I found the 1.1.6 is the latest for 8.10 too. I was looking for an excuse to upgrade, but this won't be it.
I also found that others were having this problem on Ubuntu and that in theory there was a fix in 1.1.7. When I went to sourceforge I found 1.1.7 was there and .debs existed. I didn't mess those and just got the source. At the end I created my own .deb. I got the source because its a video based app and I just wanted to do it. If you try the new .deb and it works, let me know.
I'm going to assume that you have build-essentials and checkinstall as you would to compile any programs on Ubuntu. That's all on Ubuntu's site.
I download and unpacked the source. tar -xzvf xvidcap-1.1.7.tar.gz and cd to the new directory.
Then I ran configure like normal with these flags.
./configure CFLAGS="-march=native -O2 -pipe" MAKEOPTS="-j5" -prefix=/usr
You need gcc 4.2 or higher which is on Ubuntu 8.04 to use the -march=native. Basically it will pick the best opts for your architecture, which wasn't that hard, but this is easier. I went with 02 hoping for better performance. My guess is that it would have already been compiled with this by default but I'm not sure. I have a quad so I used -j5 or cpus + 1.
When it was done I still did a make -j5. It compiled so fast I thought it didn't work.
When I tested it in src it didn't work. I wasn't happy, but it looked like a path problem looking for a shared lib, so I did the install anyway.
I ran sudo make install and all went well. I tested the program and it was working well. Better than I expected using mp4 DIVX at 24 fps and getting them all. My test was capturing a youtube video playing on my screen.

When that was done I still ran

sudo checkinstall -D make install

which reinstalled and made the .deb package. xvidcap 1.1.7-1 is now showing in my Snaptic package manager so I can uninstall it if I want later.

I think you can run checkinstall before the sudo make install and then run dpkg -i package.deb but I have never done it that way. maybe some day. Anyway xvidcap 1.1.7 is running on my Ubuntu 8.04 very well as far as I can see.

BTW - I was not using sound capture.
Good luck and let me know if you have success or failures with this.

Friday, February 6, 2009

Running Visual Studio and Vista on Ubuntu



I installed Virtualbox today with Vista. It installed by following the basic instructions with no issues. I am using the binary edition for personal use only. I simply added their source to my apt/source.list and installed virtualbox-2.1. Created a new virtual machine and put the Vista disk in my DVD. It all loaded just like a fresh install. I did make the mistake of not making my hard drive big enough, so I'll have to reinstall to get to use the new larger drive. You can't make the drive larger later.
Once I loaded Vista, I was able to turn on 3D support for my video card and networking worked right away. The screen resolution was bad in the beginning but I followed the instruction to install the additional tools and I could resize my screen as needed. I included a screen shot using the seemless mode, where the Vista and Linux windows appear side by side.
My only disappoint was in the seemless mode alt-tab uses either window or Linux but not both. Hardly a great complaint. At this point I was just pretty happy that it was working. I only set the RAM at 1024 Meg which I thought would be to low but it wasn't. You can change this later, so that's not too big a deal.
Seemless mode was interesting but the system performed better when in windowed mode. So while it looked neat to try out since alt-tab doesn't work, I might as well have a desktop and the window performance.
For a free tool I thought this was great. The personal use edition has a few features I still need to try. They say it will load VMware VM's. I couldn't get that to work with an older W2K VM I had. They also say you can install the 64-bit version of an OS as a guest on a 32-bit host to try it out. So you know I have to install 64-bit Ubuntu this weekend!

mod_rails on ubuntu

It didn't take too long but I did get mod_rails running on Ubunutu 8.04. The instructions on the Phusion site are adequate and don't need to be repeated here, but a few things are left out that I got a little hung up on.
First for Ubunutu the configurations typically go in etc/apache2/conf.d, so I created a new passenger.conf and put in there the LoadModule and other directives they said in the install script. Then I thought I was ready to go, but it didn't work.
I was converting from Mongrel and had proxy statements left over in my virtual host config. I took these out reload and the site came up, but I couldn't log in.
One last item to note. I couldn't connect to my database. I noticed rails was logging to production and not development.
In passenger.conf I added a line to set RailsEnv development. Just add that to the bottom of the config if this is your dev box. For your production box, the default is production but you might want to set it to be clear.
I ran my first load test on our application and I was impressed. The application is serving as many transactions per minute as with mongrel it takes up some memory pretty quickly to do it, but it does release it without crashing the server and that is good thing.

Skype on Linux

I was using Skype on my mac because it had a microphone and cam built in but it was getting sort of anoying to be on my linux desktop and need to have my laptop to skype, so I broke down and got a logitech headset for use with Skype on Linux. Skype started like a champ. I added the skype repository to my apt-source in etc so I could just use apt-get to do the install and get updates.
My skype out call went pretty good except I noticed the mic volume was low. I thought it was Skype but it was a simple fix.
I opened gnome-volume-control from shell or double click the volume settings. That opens the control panel. On mine microphone was not on there. I had to go to preferences and add microphone and also add Mic Boost. Then I could adjust those just fine. Not a skype problem at all.

Thursday, February 5, 2009

Mongrel 1.1.5 on ubuntu 8.04 and rail 2.1.1

We host our production environment using Ubuntu 7.04 and mongrel 1.1.4 for our Rails applications. We setup our test server with Ubuntu 8.04 and mongrel 1.1.5 and noticed the server was crashing all the time. As I started to dig into this, I noticed that mongrel was chewing up all the memory with haste. We then turned off all the rails apps and just let mongrel go for a couple of days. It ate up all the memory until it killed our VPS. If we shutdown mongrel all the memory comes right back. Just to repeat, this application has no usage. The ports are not even open. Its just a rails stack with Mongrel running and its eating all the memory. So when the guys online tell you its not mongrel, don't believe it.


So I started working on some upgrades to make sure we were at the latest versions.

I updated Ubuntu first by doing a simple:
sudo apt-get update
sudo apt-get upgrade

We did have several packages that needed to be upgraded. This ended in ruby being at version:
ruby 1.8.6 (2007-09-24 patchlevel 111) [i486-linux]

Then I needed to upgrade rubygems from 1.2.0 to 1.3.1
I couldn't use
gem update --system
I used these instead:
sudo gem install rubygems-update

sudo update_rubygems


Mongrel 1.1.5 is the latest gem I could see but I did a gem install mongrel just for fun. It recompiled and installed 1.1.5

I then did:
gem install rails
That brought me to rails version 2.2.2.
I next did a gem update for the rest of the gems. I wasn't really interested in doing all of those one at a time. There were quite a few. They shouldn't matter, because the application is not getting used at all. But if you want to see here is what was updated:
Gems updated: RedCloth, ZenTest, net-ssh, capistrano, columnize, fastercsv, highline, rubyforge, hoe, libxml-ruby, memcache-client, net-scp, net-sftp, net-ssh-gateway, packet, rmagick, ruby-debug-base, ruby-debug


To start my simple scenario I started two mongrels and let them sit. After mongrel was started the system had 102,260K free with both mongrels taking 9.4% of the memory. Now we wait.
At 1 hr of no usage we are down to 100,028K of memory. The question is when will it stop? The answer that is usually prescribed is to use Monit and kill the mongrels and restart them. No problem. I prefer to not think restarting a process is a solution to the problem, while its a fix, its not ideal.

Update: At the end of day 1 memory is now at 31456K. Now killing process.

My answer for right now is to switch to mod_rails. mod_rails has been shown in performance tests to beat 10 mongrels. I'll let you know the process to get rid of Mongrel. I understand that 37 Signals has now switched to mod_rails as well.

Let me know your experience with either of these tools.

Wednesday, February 4, 2009

Hello World

I really just wanted to start recording my usage of Ubuntu, OSX and my iPhone. Much of the content will be so that I can remember what I did, and hopefully I can give a little back to community which has helped me so much.
I think it is important when discussing Linux to make sure you are aware of the physical setup of the machine, because your mileage can vary with Linux and different hardware. Currently I'm running Ubuntu 8.04 on Intel Quad with 4GB. I boot from a SATA drive but also have software RAID 5 that I used for a long time before SATA drives were invented. I use a TV card, but because I have ATT cable now I can't get any basic channels as of yet. I'll be trying to figure out how to do this again. I recently purchased nVidia 9600GT to run my dual display. I'll post my experience getting that to work. I made a classic mistake and bought a Lexmark printer before checking online if I could use it with Linux. I can't. But it works with my mac just fine, probably have to figure out if I can share that in some way. I normally only buy HP because of their support for Linux but my last HP broke in just over a year and was hardly ever used. That ticked me off so I have a different brand now.
I think a site that basically listed the ideal hardware maybe configurations of that hardware for Linux would be great. Maybe people could upload their configs, their distro and any issues they had so that buyers could easily buy the right hardware. Links to online shopping might help fund the hosting of the site.

I'm currently working on a few projects. I am installing and working with Pentaho as an end user tool. I'd like to find some contracting work with Pentaho in the short term. I'll also be setting up VMWare as soon as I get an XP license. Sadly I have two in use, but need another one. I have an extra Vista license, but I don't want to use Vista. I'll be using some video and graphics software on Linux to see how those projects compare to my OSX versions.

In general, I'm a Unix guy. I have well over 10 years of experience compiling kernels but at that point its time to stop counting. Gentoo is my favorite distro. Unfortunately, it really fit my needs as a hobbyist but I could no longer justify using it for my production work environment. It has turned out that it is just easier to support Ubuntu and Ubuntu has run very well for us in production. Their desktop version is great, its a breeze to install and maintain. Then we deploy to the server version and everything is usually the same. I use my Mac as my family computer. My loves it, and its easy to support at home. Some of the developers use OSX for development as well, and I do sometimes. We have not had issues developing on OSX and deploying to Ubuntu yet. We have been writing Rails application primarily and have some PHP and Java work. As work on those projects I may write any articles that took me some time to figure something out.
I love my iPhone. I didn't think I would like it this much but I do. I want a ssh client to connect to my Ubuntu server for those crunch moments. Its a great phone. Nothing more to say about that.
If you have some Ubuntu questions or interesting topics. I'll see if I know the answer, just leave a comment.