Monday, 15 December 2014

Setup Debian Wheezy with 2 monitors ATI Radeon

Oh I had some trouble here...

  • With black screen problems after installing the drivers from Radeon.
  • Dragging windows to different display problems when installing drivers from Radeon. I don't know enough on how to fix this, if someone does please let me know as I would love to use the latest drivers from Radeon.

So my fix for all of this was to use the smxi tool. What a superb set of scripts!! Really saved me days of messing about. Just take a look at the documentation on the site, it's quite straight forward and it guides you through everything nicely.

You can also use this tool for a few other things apart from setting up your graphics card. I used it for that mainly.

When it came down to the options you could select the latest or previous versions from AMD, or select the Debian fglrx.

I tried the latest from AMD and got so far with that. It definitely stopped my black screen on startup but then I had a dragging window problem between monitors :-( which I don't know how to fix yet.

I just stuck with the debian fglrx install and that worked superbly. I just had to run amdcccle afterwards to just configure the monitor sizes correctly and also set the overscan on my second monitor so it removed the black border.

And that was it!

Hope this helps someone else.

Sunday, 10 November 2013

Debian Wheezy - How to remove annoying shutdown beep

I found many solution out there to turn this annoying system speaker beep off, but none seemed to work for me. Eventually I found a little something in amongst the thread.

What fixed it for me was a combination of 2 things, one was to remove the "pcskr" module (although I found this the softer of the system beeps) and the other was to go into alsamixer and select the sound card that contained the "Beep" and mute it. Yes "mute it" setting it to "00" does not work. To mute this select it and press 'm'.

So a quick summary of exactly what I did...

  • Add "blacklist pcspkr" to a blacklist.conf in the /etc/modprobe.d/ directory. This will prevent the pcspkr being loaded on start up.
  • Start "alsamixer" in a terminal, select the sound card (Press F6) that contains the "Beep" (you may have more than one listed and maybe more than one with a "Beep" in this case I am not sure what to do, but try muting one at a time to see what works or maybe both). To mute select the "Beep" and press 'm' you should see the "Beep" muted. Press Esc to exit and it should be saved.
  • You can now remove the pcspkr before you restart by typing as root "modprobe -r pcspkr", this will remove the module now before you reboot.
And that should be it :-)

Saturday, 1 June 2013

TL-WN821N with RTL8192CU for Fedora 18

Well I had a little bit of trouble with this wireless dongle on Fedora 18 but with the help of a few links all seems to have gone well. What I seemed to find out is the current default driver does not work so well and you need to get the driver source, blacklist the default drivers, compile the new drivers and install the module. Let me explain with a few easy steps...

  • Possibly you will need the kernel-source, kernel-headers and gc++ installed prior to the following instructions. 
  • Get the latest drivers from the following link, download the driver Linux Kernel 2.6.18~2.6.38 and kernel 3.0.8.
  • Blacklist some of the current rtl drivers, by adding the following to the /etc/modprobe.d/blacklist.conf 
    • blacklist rtl8192cu, 
    • blacklist rtl8192c_common, 
    • blacklist rtlwifi

  • Just doing an install.sh of the driver didn't quite cut it, I needed to apply a patch for it to compile, I used the 2 attachments found on this link from the user tim_philips.
  • I downloaded both attachments, potentially you need an account to download them but I am unsure.
  • put the download of the driver, the txt and sh file in the same folder.
  • Run run in terminal as root user: bash ./patch_rtl8188C_8192C_usb_linux_v3.4.4_4749.20121105.sh
  • If you want to know what the script does, you can take a look, it just unzips a couple of things applies the patch and makes and installs the driver.
  • One other thing if yo update your kernel, chances are you need to redo all of this, doesn't take 2 minutes :-)
That should be everything. Hope this helps. Potentially you need a reboot just to remove the currently installed drivers or you can take a look for them using lsmod and doing an rmmod <with the module names that you added to the blacklist>

Here are some helpful links...

Saturday, 18 May 2013

Updating QNAP firmware and using openssh

Just a quickie...

If you are one of the people who wants a more configurable ssh connection to your QNAP NAS box and have configured the use of openssh like from one of my previous blog posts and you notice that after a firmware upgrade of the qnap NAS the ssh doesn't seem to allow connections any more. This is usually with a "Connection Reset by Peer" error.

In this case it means that the openssh sshd has been overwritten by the firmware upgrade.

Dont Panic! Its an easy fix, you just need to re-copy the openssh sshd back over the default one on the NAS and restart your sshd either in the terminal connection to the NAS box or through the web administration tool.

So looking at James Lick's post again, its just the following 2 lines after connecting to your NAS box via telnet (well that is what I use at least):

  • mv /usr/sbin/sshd /usr/sbin/sshd-orig
  • cp /opt/sbin/sshd /usr/sbin
Now a quick restart of your sshd, this can be found in the web admin tool, 
  • Network Services, 
  • Telnet/SSH,
  • Click Allow SSH connection to disable it then apply
  • When its disabled, re select it to enable it and then press apply.

That should get your ssh back up and running.

Saturday, 2 March 2013

Setting up printing and scanning of hp photosmart C310 printer over network with fedora 18

Here is what I did.

I installed the hplip packages: (sorry I don't know if I added any extra repository, but if you don't see it the only repositories I have added are the "rpmfusion free" and "rpmfusion free -updates")
  • HP Linux Imaging and Printing Project
  • Files needed by the HPLIP printer and scanner drivers
  • HPLIP Graphical tools
  • HPLIP Libraries
Once I installed them I opened up Hp Device Manager (I pressed the search app button i.e. windows button).

Once open...
  • Selected Network/Ethernet/Wireless netowrk
  • Click Show Advanced Options
  • Select Manual Discovery
  • Type in IP address of printer (you can find this on printer press the wireless button on the display, the ip address should be listed, usually something like 192.168.0.<Something>)
  • Click Next
  • Your device should appear in the list and selected.
  • Click Next
  • Click Add Printer
  • You will probably need to enter your root/admin user name and password
Now the printer should be installed and also the scanner. 

You can check that the scanner is installed by using terminal and entering "scanimage -L" this will show you all scanning devices possibly webcams also.

You should be able to start up the scanning software and start scanning.

This link helped me :-)

Saturday, 2 February 2013

Using Oracle Java with libreoffice Fedora 18

Problem I was having was that I didn't want openjdk installed as I need Oracle's Java environment to be able to develop some android apps. I am not fully sure if I needed to remove the installation of the openjdk I guess I didn't and all I needed to do was to update the links and environment variables to point to the installation of Oracles Java. Anyhow I removed the openjdk instead and this is how I did it.

What I did here was to remove the installation of the openjdk without it trying to remove the libreoffice using following command. Use the appropriate version of the openjdk to remove it.

rpm -ev --nodeps java-1.7.0-openjdk

The --nodeps will remove the package without the libreoffice.

Now I installed Oracle's Java rpm downloaded from JavaSE Download, just by double clicking it, it then installs it. After completion of the installation, libreoffice seems fine and so does my development environment.

You can check which version of java is wired up if your not sure, by using the command:

java -version

When I was trying to do this I managed to get myself into a pickle with versions of java being installed, as I couldn't remove Oracle's Java using the software tool in Fedora 18. The software tool seemed to crash all the time, so I removed Oracle's Java from the command line instead, I used the following commands:

rpm -qa | grep jdk                            (This to find out the name of the package I wanted to remove)
rpm -ev jdk-1.7.0_13-fcs.x86_64    (This to remove the above package)

for some reason the software tool wasn't updated with the removal of the package but I ignored it for now.
So I found I had to do the order of removing the openjdk package, which I believe removes all the environment for java, so then installing Oracle's Java would reset them and then libreoffice continued to work.

I also imagine if there are any updates in the future that require openjdk to be installed it will be installed and thus wiping your Oracles Java environment variables. I haven't found this yet, but I am sure all I would need to do to reset them is too re-follow the above.

Remove openJDK, Remove Oracle Java, Re-Install Oracle Java. 

Hope this helps someone else too. :-)

Sunday, 27 January 2013

Fedora 17 and 18 headphone socket fix

The Problem...

My headphones produce no sound from jack socket when using Fedora 17 and also Fedora 18. The sounds card I have is:

  • 00:1b.0 Audio device: Intel Corporation 5 Series/3400 Series Chipset High Definition Audio (rev 06)
  • 02:00.1 Audio device: Advanced Micro Devices [AMD] nee ATI Redwood HDMI Audio [Radeon HD 5000 Series]

I found the above be executing the "lspci" command, you can use "lspci | grep Audio" and that should find similar to the above.


The Solution...

I got my solution from here.
All I did was to create the file /etc/modprobe.d/dist-alsa.conf and add the following lines:

  • options snd-hda-intel model=dell-m6-amic
  • options snd-hda-intel enable_msi=1
Ok I found an issue when I tried the above setting of setting the model to dell-m6-amic, my microphone stopped working, I then found a little information on this thread that stated that was for the analog mic, so I changed the setting to dell-m6-dmic which is for a digital mic.

How to know which model to choose this link is very useful! This has some info too but pretty much the same.

Then reboot, potentially you don't need to reboot and there is a command to restart the sound system, I am not sure on this, so reboot it was for me :-)

Hope this helps someone else out too :-)

Thursday, 24 January 2013

Installing ATI drivers on Fedora 18

I followed mostly what was in this link, but when installing the driver I just did yum install akmod-catalyst as it pulls in required dependencies.

I managed to get a black screen by just installing the akmod-catalyst without some of those instructions, so it was either the first instruction or last that fixed my black screen problem after booting. reinstalling mesa-libGL or the dracut of a new image after making sure the Radeon drivers were uninstalled.

Only problem I have is a nice watermark at the bottom right of my screen. Its not so bad for now, but would be nice to get rid of :-)

There are some fixes out there on how to remove it, but I am sure the new builds releases will come soon. :-)

Wednesday, 23 January 2013

Nice shortcut editor for Fedora 18

I used alacarte and seems to be able to set up all the shortcuts and modify the icons etc. Very nice! straight forward.

yum install alacarte

Setting up Fedora 18 for an SSD

Just some short points on setting up Fedora 18 or for that matter probably most Linux flavours for an SSD.

I've highlighted the bits I changed:

/etc/fstab changes:


  • /dev/sda   /   ext4   noatime,nodiratime,discard,errors=remount-ro 0 1
  • tmpfs   /tmp       tmpfs   defaults,noatime,mode=1777   0  0
  • tmpfs   /var/spool tmpfs   defaults,noatime,mode=1777   0  0
  • tmpfs   /var/tmp   tmpfs   defaults,noatime,mode=1777   0  0
  • tmpfs   /var/log   tmpfs   defaults,noatime,mode=0755   0  0 (only do this if you don't care about loosing logs on reboot!)
/etc/default/grub changes:
  • GRUB_CMDLINE_LINUX_DEFAULT="quiet splash elevator=deadline" add the elevator bit to the end.
  • grub2-mkconfig -o /boot/grub2/grub.cfg
Chrome to use its cache directory as the tmp director, which is in memory already add the following to the shortcut.
  • --disk-cache-dir="/tmp" --disk-cache-size=104857600

The link I got most this information from :-)

Setting up Broadcom 4312 with Fedora 18

This was a bit of a pain and felt like I had to jump through hoops to get this damn thing to work. In the end its not actually too much to do, you can either to try and follow the instructions in this link (I have had to do this recently) or mine.

First off, by using the command line rpm and yum commands I seemed to be more successful installing all the required packages.

So here goes...

Step 1:
Change user as root:
  • su -
Step 2:
You need to add a couple of repos, the rpmfusion free and nonfree repos.
  • rpm -Uvh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm
  • rpm -Uvh http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm
Step 3:
You will probably then need to download and import the keys for these repos from rpmfusion-keys:
  • get [attachment:RPM-GPG-KEY-rpmfusion-free-fedora-18]
  • get [attachment:RPM-GPG-KEY-rpmfusion-nonfree-fedora-18]
Step 4:
Import the keys:
  • rpm --import <Directory where you downloaded the [attachment:RPM-GPG-KEY-rpmfusion-free-fedora-18] Key>
  • rpm --import <Directory where you downloaded the [attachment:RPM-GPG-KEY-rpmfusion-nonfree-fedora-18] Key>
Step 5:
Now with these keys imported, you should now be able to update:
  • yum update
Step 6:
Ok this got a bit weird when I tried to install kmod-wl but try it out it could work straight out for you, but I had issues with it failing to install due to "uname -r" not matching with build of kmod-wl not correct.
  • yum install kmod-wl
If the above fails then try:
  • yum install akmod-wl
I think all the akmod-wl will do is bring all the dependencies it requires, which will be the kmod-wl and the broadcom-wl packages and a few others. I think all the akmod does is to be able to build a new version of the kmod-wl module against your version of the kernel if there doesn't exist one on the repos. Also if/when you update your kernel this akmod should make sure that the kmod-wl is built correctly.

So at this point akmod-wl, kmod-wl and broadcom-wl should be installed, and a few others required dependencies by the akmod-wl package.

Step 7:
Ok now I found at this point that there was a module loaded called ssb which conflicts with the wl module. I am not sure what this module is and why it conflicts I didn't look into it too much, so I had to remove this module permanently by adding it to a blacklist. First of all I tried:
  • echo "blacklist ssb" >> /etc/modprobe.d/blacklist.conf
which didn't seem to work, so I added it directly to the grub.conf, but what I did was add it too the /etc/default/grub file so as the time comes when you may need to call grub-mkconfig then it will keep the default added in that file, so I did:
  • added "rd.driver.blacklist=ssb" to the end of the GRUB_CMDLINE_LINUX in the /etc/default/grub
  • then ran, "grub2-mkconfig -o /boot/grub2/grub.cfg" this regenerated the /etc/grub2.cfg file.
I am not sure why I had to blacklist like this and if anyone can explain or help out that would be much appreciated :-) So all should be prepared at this point, just one more step.

Step 8:
I removed the ssb module and added the wl module.
  • rmmod ssb
  • modprobe wl
Now that should be it, as far as I remember, on the network icon then should show up wireless. Try rebooting also to make sure it comes up after a reboot also. If you get any problems or some parts don't work please let me know and I will try to correct it and help.

Some useful links I found and used.
Hope this helps out :-)

Saturday, 19 January 2013

Setting up QNAP to accept ssh connection from different users

This is a pain to do, and I always seem to forget one switch to turn on 1 file to set up 1 permission to configure. Always the same every time I need to do it. Well I have put together a few links that I have found useful and a few procedures I have found helpful to understand why a connection fails. :-)

So the steps and links I used are:

  1. For setting up new ssh software on the QNAP NAS I used this link: http://blog.jameslick.com/?p=1090 brilliant, excellent! worked for me and my model. I imagine it would be a slight tweak for other models if they have different names for equivalent mounts.
  2. Then I set up the users paths in /etc/passwd on the NAS box of the users.
  3. Setting up the ssh keys, omg! I used this link: http://superuser.com/questions/8077/how-do-i-set-up-ssh-so-i-dont-have-to-type-my-password. So I worked out the public key goes on the NAS box in the authorized_keys file in the user .ssh directory of the user, e.g. /home/JoeBloggs/.ssh/authorized_keys  <-- This user home path should be the same as what you specified in the /etc/passwd file. BE CAREFUL - When copying the public key that it doesn't gain spaces on the copy paste route, it needs to be as it is in the generated file (from point 1) You can add more than one key to this file for this user, if the same user connects from different machines, you can generate another key from the other machine and add the public key of that one to the same authorized_keys file. You press ENTER after the first one and then paste in the new one.
  4. Configure the sshd_config file this is in the location(s) specified in the link 1, don't forget to update both configs otherwise on reboot you will get your old one back. You can set up the config file with the users that are required to access via ssh.
    1. So there is a section to describe where to find the authorized keys file, and you can configure it to go to the home path of the user that tries to connect, using (I believe %h means the home directory of the user connecting, but I am not 100% sure): 
      • AuthorizedKeysFile %h/.ssh/authorized_keys
    2. There is also a section to allow certain users: 
      • AllowUsers JoeBloggs Fred
  5. There are also special permissions allowed only to the authorized_keys files which I found in this link: http://recursive-design.com/blog/2010/09/14/ssh-authentication-refused/
  6. Any problems that occur, when you test to connect to ssh JoeBloggs@<yourIP> you can run the ssh with a -v that can output much information that can give clues in why it fails.
  7. Also you can run the QNAP sshd manually with sshd -d and then connect from the client and watch the debug output that can give clues also :-). To do this I would stop the ssh service in the QNAP web client. Then locate the sshd on the QNAP, it is usually at /usr/sbin/sshd, so you will have to execute with full path specified e.g. /usr/sbin/sshd -d and this should then start up the daemon ready for connections.
  8. One more thing! If you want to map a network drive on a linux box to your QNAP, you will need to turn on the sftp option when turning on your ssh option in the web client of your QNAP box. You can also do this drive mapping nicely with sshfs JoeBloggs@<YourIP>:<Directory you want to go too on the QNAP> <Directory you want to map too on local computer> The "Directory you want to go too on the QNAP" default is your home directory of that user, so if you need somewhere out of that (and make sure you have permisions for the user set up on the QNAP), you have to specify a relative path from your home directory, e.g. sshfs JoeBloggs@<YourIP>:../../SomeSpecialData /home/JoeBloggs/SomeSpecialData
Ok I think that's it for now.

Hope it will help someone, again it will help me when I need to do this again :-)

Migrating QNAP NAS from Raid 1 to Raid 5


What a mess!!

BACKUP FIRST!! Somehow I didn't even need it in the end :-)

Well, I think I have eventually managed to work out how to get my raid migrated from Raid 1 to Raid 5 after adding a new drive (surprisingly without loosing any data), and let me tell you something I would feel sorry for the average guy who doesn't know too much about Linux or computers to be able to do this on the command line. I feel that the UI isn't up to coping with a situation that something doesn't go correctly too well. This was not a straight forward thing to do and probably the first thing I would do is to never use the UI provided to do this, its next to useless! No really! the best thing to do is to learn how to do it manually from the command line, I had a lot more confidence working with Linux directly over this UI. 

It has taken me a couple of weeks and I have my NAS back up and running at Raid level 5 with the new drive and over 50 emails back and fore to support which was very helpful and responsive and gave many clues but in the end I worked out the bulk of it myself, which gave me a great sense of achievement and knowledge. 
I think I rebuilt Raid 1, 3 - 4 times, Raid 5 twice, while removing and trying to re-add my new drive, also scanning drives for bad blocks and as they are all 2 TB drives this doesn't give results back that quickly. So now I am going to try and document what i did to get this to work with the help of a few links one of which I was guided to by the QNAP Support.

So the useful links I had were the following:
Thread 1: viewtopic.php?t=12109

This was the most useful one for clues on the process of what I needed to do, I got so far with this one and failed to unmount the drive as there were services still being used so I needed to run "/etc/init.d/services.sh stop" this stopped all the services I could then unmount the drive. Further down this thread from "abmagfab" he gives more information if you need to grow the raid also.
Thread 2:http://www.davelachapelle.ca/2008/07/25 ... to-raid-5/

This one gave more more information on how to convert the Raid 1 to Raid 5 on 2 disks and then add the new drive. Also very useful. I think I would just use this thread next time.

The trouble with doing all this from the command line is that the UI will start to look incorrect as the files that the UI relies on does not get updated when running these commands on command line. The files were /etc/storage.conf and /etc/config/raidtab. I wasn't sure what I needed to update on these files so I kind of guessed, when I had 3 drives at Raid level 5 I set them accordingly. I tried to find more information about these files and what the different parameters meant but I couldn't, so please if anyone knows where I can find information on this please post here, I would love to understand how they are used and what the possible values to the possible parameters are. thanks  

So I would just follow thread 2 but I would also unmount the /dev/md0 also so would need to run the "services.sh stop" command.

It wasn't as straight forward as the thread as I had already started playing with mdadm and I got things in a bit of a state but I managed to get it all back to a nice state and learned a lot on the way (surprisingly without loosing data too!)

I had a situation once where all was complete, but my /dev/md0 would not automount I had forgotten to turn the journaling back on with the /dev/md0 (quite easily done when you have rebuild times over night and day, I kind of forget what I did! ha ha) once I did that and rebooted, voila! auto mounted!

dmesg is your friend, it can give quite a few clues, I will explain how I found the journaling off problem...When I ran dmesg straight after boot and after I manually mounted /dev/md0 (mount /dev/md0 /share/MD0_DATA -t ext4) it gave me the following on the last line of dmesg:
EXT4-fs (md0): mounted filesystem without journal
I saw the "without journal" and that triggered my memory about whether i turned it off or not and a quick search online of what happens when mounting without journal, gave me some more clues so I just turned it back on using command in Link1 and all seemed to be fine again 

Don't ask me how I didn't loose any data, but I didn't (well not that I have noticed yet!). mdadm is very robust and usable just make sure to read the man pages for all the options that you are about to use from any links that you follow and you can learn a lot just like me 

I am sure it will help me in the future when I forget all this in a few months (well few minutes really)

I hope this helps someone else.

Wednesday, 12 December 2012

Bananas and Monkeys


Ok first off, I did not write this, but damn it has some truth. For acknowledgement here is the link I found it on. It goes something like this...


Start with a cage containing five monkeys.
Inside the cage, hang a banana on a string and place a set of stairs under it. Before long, a monkey will go to the stairs and start to climb towards the banana. As soon as he touches the stairs, spray all of the other monkeys with cold water.
After a while, another monkey makes an attempt with the same result - all the other monkeys are sprayed with cold water. Pretty soon, when another monkey tries to climb the stairs, the other monkeys will try to prevent it.
Now, put away the cold water. Remove one monkey from the cage and replace it with a new one. The new monkey sees the banana and wants to climb the stairs. To his surprise and horror, all of the other monkeys attack him.
After another attempt and attack, he knows that if he tries to climb the stairs, he will be assaulted.
Next, remove another of the original five monkeys and replace it with a new one. The newcomer goes to the stairs and is attacked. The previous newcomer takes part in the punishment with enthusiasm! Likewise, replace a third original monkey with a new one, then a fourth, then the fifth. Every time the newest monkey takes to the stairs, he is attacked.
Most of the monkeys that are beating him have no idea why they were not permitted to climb the stairs or why they are participating in the beating of the newest monkey.
After replacing all the original monkeys, none of the remaining monkeys have ever been sprayed with cold water. Nevertheless, no monkey ever again approaches the stairs to try for the banana. Why not? Because as far as they know that's the way it's always been done round here.
And that, my friends, is how company policies are made.


Interesting huh?

Tuesday, 4 December 2012

NHibernate, Ninject 3, MVC 4

Recently I have started to use Ninject more and more, its my favourite of all the IOC frameworks I have used so far. Also I wanted to give MVC 4 a dabble just to keep myself in the loop with what is going on in the MVC world after only using MVC 2 in the past. So I thought I would try to wire up something nice between MVC4, Ninject 3 and NHibernate and came up with the following located on github here.

So in a previous post with NHibernate, Unit of Work, Repository and Structuremap, I carped on about a Unit of Work and a Repository.


With regards to the repository...

I think I have gone off the idea of using the Repository pattern with NHibernate after reading lots of posts on Ayende's site, various other sites and from previous experience I feel it is limiting NHibernate to make optimal calls to the database. I believe NHibernate's Session is the repository. Here are a couple of links:
Architecting in the pit of doom: The evils of the repository abstraction layer
Repository is the new singleton pattern


With regards to the Unit of Work...

Here I just set up a filter to wrap up the work done across a transaction. So now any of the work in the domain model/business logic that is done via a particular controller action should all be wrapped up in a transaction. I wired up the filter using the Ninject.MVC3 extension and the Kernel.BindFilter.


Github project...

Its been a while since I looked at this project and what I was doing, so I will need to refresh myself with it as well. I just created the default MVC 4 app and added a new controller, the ProductController. I just wanted to be able to show a list of Products that are stored in the database and also be able to add new products and render the change.

Libraries I used:
MVC 4
Ninject 3
Ninject.MVC 3 extension
NHibernate
Automapper

The project isn't finished yet, I would like to try to wire up NServiceBus and RavenDB in here too (but these are just distance thoughts for the moment), and really push all the business logic down into the domain model.

Some points on what I did.
  • Created the standard ASP.NET MVC 4 application. 
  • Added NHibernate and created a simple database relationship for stores, products and employees, I may have taken these relationships from somewhere else but I can't remember.
  • Added Ninject and its Ninject.MVC 3 extension, this works for MVC 4.
That's it for now... I will come back and update this some more when I get a chance...

Monday, 26 November 2012

Programming is not just programming

Well its been a very long time since I posted something here, but today I had an idea of what I could write about...

I had a very good friend ask me about .net development, and that he was concerned about getting his .net skills up to scratch. My immediate thought was this guy is a super bright guy and I am sure he knows enough of .net to be able to do pretty much anything that was required of him and quickly as well. Which then led me onto thinking of this post and what I could write about.

I would say that a lot of .net developers out there know how to write code in .net and probably/possibly many other languages too, but possibly miss the idea of writing code that can be maintained and also feel confident about being maintained quickly (this does not mean that the super bright guy above does not know this :-)).

I would say of those that don't care and don't even realise what they are writing, their code is becoming legacy code as quickly as they type it. I for one was one of these guys (the didn't realise one), absolutely 100% omg! Am I admitting that! We all have to start somewhere (that's my excuse ;-) ) and from there its a lot of practice and realisation of what we are writing.

Now I am making a concentrated effort on everything I write to be much clearer, simpler, easier to understand and change, by keeping in mind as many of the things I have read about in the following books, links, blog posts frameworks etc.

I will try to keep this up to date every time I discover something new or useful. Lots of links and lots to digest I know, but it has taken me some time. Just take your time, practice a few things, try them out on some scratch projects see what you can come up with. You'll surprise yourself on how your code becomes much more awesome!

  • S.O.L.I.D - A must for every OO developer, probably any other link on that page too.
  • Tell Dont Ask - Another nice one that can lead you onto the other nice things like CQRS and Law of Demeter
  • Dependency Injection Stack Overflow link and James Shore Link from the same SO link - Lots of Inversion of Control (IOC) containers out there to help with this too. Ninject (my favourite), StructureMap, Unity, there are also more. I find this is really important and should be used on every project regardless of size of the project, what you will start to realise when you use this that your unit testing becomes easier to write.
  • Unit Testing - There are many unit test frameworks out there to pick from, NUnit (I have used a lot), XUnit (but heard a lot of good things about this and I think I will start to use this for some home projects ), MSTest (Microsoft's, included with Visual Studio). Just a quick side note I would like to write an article about is Unit Testing. I believe and found it so difficult to start writing unit tests, this is also a lot of practice and when you realise how it fits and what it does, BOOM! It will be a Eureka moment! 
  • Resharper, get it! It's worth it! It has some very nice tools with it, the solution wide analysis(dont need to waste time building solution to find errors), the stack trace viewer are a couple. I recommend learning a shortcut a day, and within no time you wont need your mouse to code. Pffff, the amount of times I have sat there with someone using the mouse to click on menus, scrolling around, switching windows, looking for code in a solution, blah blah, all that time wasted.... aaarrrrgggghhhhh! Drives me nuts! Mind you I can get some 5 second power naps, when they are searching for the next thing to click! lol ;-)
  • Read blogs articles, follow some recognised people, like some of the people from the books I have mentioned. Also take a look at some code you have found on github, contribute maybe too, you will learn so much like this.
  • Go to some talks, organised day events, local newsgroups in your area, etc. skillsmatter, DDD
Ok some other stuff, buzz words, people, software etc... I will try to write these more in depth at some point but it will be worth you guys having a sniff at what they are about also.


Continuous Integration

Setting up a build and release process just off a single check in! If you can get this right, this is one of the most powerful things in releasing software fast and reducing all that wasted time, matching up bugs with changesets, with releases, deploying the correct version for your testers, unit testing, integration testing etc. A couple more pieces of software provided from JetBrains, TeamCity and  YouTrack is worth taking a look at also, extremely useful for tracking all task, bugs etc. There are many other solutions out there too, but these are the first that came to mind for the moment.

Source Control

There are several source control systems out there, here are a few...
  • GIT - Extremely powerful and gives much more ease and freedom for your code. I have only started to use this recently, it takes a little time to get your head around if you are used to using a more centralised system, here are some links... GIT Succinctly, GitHub get an account, try it out, practice, stick at it. Well worth it!
  • TFS - Microsoft's version of source control, along with managing tasks and probably other things too. I use it most of my working time. But the more I use GIT the less I want to use this.
  • SVN - Not used in a long time and used sporadically.
  • SourceSafe - This has to get a mention! DO NOT USE IT!! REALLY! DON'T! Go take a read of any articles about it, see what people have to say about it, maybe you have had your own experiences with it...


Database Stuff

Apart from the usual same old boring databases are databases blurb, blah blah, same old, same old, SQL Server, MySQL, PostgreSQL, blah blah etc ooops forgot about Oracle.
But there are these new document databases turning up, which seem very very interesting, take a look, try them out, RavenDB and MongoDB. I haven't used them yet, but will be giving them a go soon I hope. They seem extremely powerful and fast. I went to a see Oren Eini and he gave an excellent talk on RavenDB. Interesting thoughts of why now we should be looking at storing and retrieving data differently.


Books

  • Working Effectively With Legacy Code - Michael Feathers, Don't be depressed with crap code, take a read of this and see how you can fix it! Very good read and interesting.
  • Clean Code, and The Clean Coder - Robert C. Martin (Uncle Bob), Both these books very nice.
  • Refactoring: Improving the Design of Existing Code. Excellent! Nice techniques, mixing this with Michael Feathers book above and the Clean Coding books of Uncle Bob you will have some quite powerful ways of being able to modify the most complex of code.
  • The Pragmatic Programmer: From Journeyman to Master - Andrew Hunt and David Thomas,  Superb, must read for all developers! Interesting too, see if you can recognise any of your situations you have been in match whats in this book, I wouldn't be surprised most of them match :-)
  • The Art of Unit Testing - Roy Osherove, Very nice, a definite good start with Unit Testing, but I still think you need to practice and see what it really is worth.
  •  C# In Depth - Jon Skeet, Superb! I have to take my time in this one, and probably have to read it a few times, but excellent!
  • Domain Drive Design: Tackling Complexity in the Heart of Software - Eric Evans, I have started reading it but not got very far yet, but heard it is very good.

Other Things

  • Rx - A must! Take a look! Try it out! See what you can do and learn from this. Very powerful!
  • Agile - I would say a very powerful, fast and useful process of writing software. I would never go back to Waterfall again if I could help it, unless I needed money for sleeping. I haven't read up much on the agile processes but been in the flow of using this style of development with several projects. 
  • NServiceBus - I haven't used yet but all I hear is good things about this piece of software check it out. I will be trying this out in the near future also.
  • Automapper - For transfering data from one object to another which has the same properties, Very neat, very useful, easy to use, set up.
  • Some people, Martin Fowler, Kent Beck, Robert C. Martin (Uncle Bob), Oren Eini, Udi Dahan, Jon Skeet, all of these have a massive community around them and well worth reading up some of their articles.

Saturday, 9 July 2011

Is Switch by definition a Code Smell?

I am fed up of coming across this kind of shite code all the time, whenever I need to fix a bug I need to change 400 things not 1 thing.

   22     public class CrapiolaManager : ICrapiolaManager
   23     {
   24         public void DoSomething(SomeEnumOfTypesWtf someType)
   25         {
   26             if (someType == SomeEnumOfTypesWtf.SomeType1)
   27             {
   28                 //1. Do some work and calculations etc and more crap code...
   29             }
   30             if (someType == SomeEnumOfTypesWtf.SomeType2)
   31             {
   32                 //2. Do some work and calculations etc and more crap code...
   33             }
   34             if (someType == SomeEnumOfTypesWtf.SomeType3)
   35             {
   36                 //3. Do some work and calculations etc and more crap code...
   37             }
   38             if (someType == SomeEnumOfTypesWtf.SomeType4)
   39             {
   40                 //4. Do some work and calculations etc and more crap code...
   41             }
   42             if (someType == SomeEnumOfTypesWtf.SomeType5)
   43             {
   44                 //5. Do some work and calculations etc and more crap code...
   45             }
   46 
   47             throw new InvalidOperationException("Invalid type passed");
   48         }
   49     }

It’s absolutely nonsense!! We are using an object orientated language ffs! Can we not see the impact of the problems this kind of code leads too?
  • Lots of bugs/issues!!!
  • New stories take longer to do!!!
  • Its duplication of code!
  • It’s not easy to Test!!
  • Not easy to read when you have to follow code through so many layers, you want a piece of logic in ONE place… Domain Model Entity?
  • It’s a problem when a new type appears! Somehow the person working on the new type needs to magically pull out of his ass some knowledge to understand where all the switch’s and if’s are in the code to alter them to add a new case! Hmmm… I wonder how many of these if’s and switch’s  get missed when adding a new type? Oooo let me guess… ALL OF THEM! It’s completely unmaintainable rubbish!
Having an interface on that manager is also completely pointless!
What’s it there for, are you going to create another manager from the same interface(highly unlikely)? Is it there for the Dependency Injection instead? Then I am sure you can inject it without an interface. Even if you’re going down this road asking yourself these questions, you may realise 'ooo' the code is in the wrong place and we shouldn’t have Managers and Helpers littered everywhere! And this is another post all in itself (Say NO to managers/Helpers)

Throwing an exception if the type doesn’t exists is also crap!
I mean seriously if you were testing this and for some reason you never hit this if/switch statement after adding your new type, that exception will never be thrown!! Oh yes and I guarantee it will be thrown when a user uses it!
  • BIG Problem!
  • Support costs!
  • Customer trust levels!
  • The list carries on…
 This code is a time bomb waiting to happen!

A suggestion to resolve this problem! (Not too much detail here)
(This is the basics of the start of the fix but should pretty much be an answer for this issue). Hmmm… how about use an interface or a base class (depending on what you require) and derive these types from that interface/base class to force implementation of the method that does the work in them dumb ass if’s and switch’s! It takes 2 minutes to write, and NOT 2 days to fix bugs when adding new types!

   22     public interface IDoSomething
   23     {
   24         void DoSomething();
   25     }
   26 
   27     public class DoSomehting1 : IDoSomething
   28     {
   29         public void DoSomething()
   30         {
   31             //1. Do some work and calculations etc and more crap code...
   32         }
   33     }
   34     public class DoSomehting2 : IDoSomething
   35     {
   36         public void DoSomething()
   37         {
   38             //2. Do some work and calculations etc and more crap code...
   39         }
   40     }
   41     public class DoSomehting3 : IDoSomething
   42     {
   43         public void DoSomething()
   44         {
   45             //3. Do some work and calculations etc and more crap code...
   46         }
   47     }
   48 ...
   49     public class DoTheWork
   50     {
   51         public void Work(IDoSomething doSomething)
   52         {
   53             //The object does the work not some external object that would be a nightmare to maintain.
   54             doSomething.DoSomething();
   55 
   56             //The above is called instead of doing...
   57             //ICrapiolaManager someManager;
   58             //someManager.DoSomething();
   59         }
   60     }

It’s not all about finishing as fast as possible and just letting your fingers typing your brain farts for 100% of the day, surprisingly it’s a very high amount of THINKING! The chances are when you come to the solution in your head after thinking around all the issues, scenarios and solutions to the problem you will find you will hardly type any code!

Here is a link I found very quickly (I am not sure who it is by) on the issue there is a lot of info here and was hoping to find something by the main man himself Martin Fowler who we should all know and love! It pretty much goes over a lot of issues with using a switch (or ifs in our case from this code, it’s the same thing) in an OO language. IMO we should be able to turn the ‘switch’ and ‘if’ statements OFF in the compiler and to be only allowed to turn them on where we need them, more than likely in the Domain Model Entities only!

I can’t stress how important something as simple as this is, It’s a simple piece of code and if thought about for at least 30 seconds, the problem could be solved much more elegantly and safely. If you’re unsure on something don’t hesitate to ask the question to a colleague. I don’t have all the answers but I can see some of them. Just by asking the question sometimes you actually answer it yourself without hearing an answer (I do that a LOT).

If this doesn’t seem to be an important issue to you and want to continue to switch and if on types I am sure there are plenty of job opportunities out there as a farmer! Let us developers write robust code and not fix someone else's problems all the time.

Monday, 7 March 2011

Nhibernate, Unit Of Work, Repository and StructureMap

This could be a long blog...

Well I have spent some time recently trying to understand how to tie all this lot together and I think I have finally cracked it in its simplest form, I am hoping at least. This is my first stab at it and I am already thinking of changing it slightly, but I want to show what I did first.
If there is anything you don't like on here or want to question please come and comment as I would love to get some feedback on what people thought of this. Problematic, see problems with it or even helped to understand it better.

NHibernate
So first of all I modelled a simple domain, just an account user and mapped this to NHibernate to do its thing with the db. Understood then that the NHibernate's SessionFactory should be once per application as its very expensive to create and that ISession is what to use every time you want to make a single or group of changes to the db, or to just retrieve anything.

Code that I Wanted/Needed
So the following are the general things I used and set up simply as possible for now.

Repository
First off I read up loads of articles on creating a repository to wrap up simple NHibernate commands. This could be extended in the future to provide a more advanced implementation. Some would argue not to have the repository at all and use the NHibernate calls directly. I like the idea of keeping the repository implementation as I am hoping it decouples NHibernate from the application (don't know why I will ever need to not use NHibernate yet but its there). I would like to be able to create as many of these as I would like although at the moment I am unsure why I would need more than 1 per thread. This will contain a UnitOfWork.

    5     class Repository : IRepository
    6     {
    7         readonly IUnitOfWork _unitOfWork;
    8 
    9         public Repository(IUnitOfWork unitOfWork)
   10         {
   11             _unitOfWork = unitOfWork;
   12         }
   13 
   14         #region IRepository<AccountUser> Members
   15 
   16         public T GetById<T>(Guid id)
   17         {
   18             return _unitOfWork.CurrentSession.Get<T>(id);
   19         }
   20 
   21         public void Add<T>(T entity)
   22         {
   23             _unitOfWork.CurrentSession.SaveOrUpdate(entity);
   24         }
   25 
   26         public void Remove<T>(T entity)
   27         {
   28             _unitOfWork.CurrentSession.Delete(entity);
   29         }
   30 
   31         #endregion
   32     }


UnitOfWork
This implements IDisposable and is what I used to manage the transaction of the NHibernate session. I would like it to dispose of the session once its been commited or rolledback, it just has a simple commit and rollback and manages to close the session. The session and transaction is created in the constructor here, so the UnitOfWork is ready to be commited or rolledback. The more I am typing here the more I am feeling this class is clunky around the edges of cleaning up. I am not sure if I need to rollback the transaction by default in the dispose if its active already, or whether the Session manages maintains that on a dispose of itself? If anyone knows let me know :-)

    6     public class UnitOfWork : IUnitOfWork
    7     {
    8         private readonly ISessionFactory _sessionFactory;
    9         private ISession _currentSession;
   10         private readonly ITransaction _currentTransaction;
   11 
   12         public UnitOfWork(ISessionFactory sessionFactory)
   13         {
   14             _sessionFactory = sessionFactory;
   15             _currentTransaction = CurrentSession.BeginTransaction();
   16         }
   17 
   18         #region IUnitOfWork Members
   19 
   20         public ISession CurrentSession
   21         {
   22             get { return _currentSession ?? (_currentSession = _sessionFactory.OpenSession()); }
   23         }
   24 
   25         protected ITransaction BeginTransaction()
   26         {
   27             return _currentSession.BeginTransaction();
   28         }
   29 
   30         public void Commit()
   31         {
   32             if (_currentTransaction.IsActive)
   33                 _currentTransaction.Commit();
   34         }
   35         public void Rollback()
   36         {
   37             if (_currentTransaction.IsActive)
   38                 _currentTransaction.Rollback();
   39         }
   40 
   41         #endregion
   42 
   43         #region IDisposable Members
   44 
   45         public void Dispose()
   46         {
   47             if (_currentSession == null) return;
   48 
   49             _currentSession.Dispose();
   50             _currentSession = null;
   51         }
   52 
   53         #endregion
   54     }

As you can probably see there is no creation of the interfaces SessionFactory or UnitOfWork in repository and this is where I have used StructureMap to help out.

StructureMap
I went the Ninject route initially as it had a cool looking site and seemed quite nice to use also. I managed to achieve what I wanted with it and it wasn't difficult at all but I wont show that here. I researched further into which DI/IOC to use and I kind of had a good feel for what people were saying about StructureMap so I changed to use StructureMap instead.
OK so I figured I needed
  • 1 SessionFactory per application running.
  • 1 UnitOfWork per thread running.
  • No limit to the number of Repositories. (not sure why this can't be 1 instance per thread also but I don't think there is harm in having more than 1 as it will use the same UnitOfWork if its in the same thread.)
    9    public static class ContainerBootstrapper
   10     {
   11         public static void BootstrapStructureMap()
   12         {
   13             ObjectFactory.Initialize(x =>
   14             {
   15                 x.ForSingletonOf<ISessionFactory>().UseSpecial(y =>
   16                     y.ConstructedBy(context =>
   17                     {
   18                         var nhConfig = Fluently
   19                          .Configure()
   20                          .Database(MsSqlConfiguration
   21                                     .MsSql2008
   22                                     .ConnectionString(connstr =>
   23                                         connstr.FromConnectionStringWithKey("db")))
   24                          .Mappings(mappings => mappings
   25                                                 .FluentMappings
   26                                                 .AddFromAssemblyOf<AccountUser>())
   27                          .BuildConfiguration();
   28 
   29                         return nhConfig.BuildSessionFactory();
   30                     }));
   31                 x.For<IUnitOfWork>().HybridHttpOrThreadLocalScoped().Use<UnitOfWork>();
   32                 x.For<IRepository>().Use<Repository>();
   33             });
   34         }
   35     }
So in here, I am setting up the
  • 1 single SessionFactory ever for the application. The x.ForSingletonOf<ISessionFactory> will sort this out. It uses the Fluent style configuration of the sessionFactory using the database connection and mappings from the domain model set up.
  • Per thread UnitOfWork, we want this as we may have more than 1 save, update and delete on the same transaction, so when its injected into the repository its the same UnitOfWork that is used before the repository has begun to be used, I will show a test later on with this.
  • Repository.
A Couple of Tests
I will try and show a couple of tests that I set up for this, and show how it could be used from the outside. I hope my tests aren't too bad, I know I find it difficult to write a good test, come slate them let me know what you would do, naming them something different? anything? I need to learn more about tests.

Test to write data to the database was successful.
   67         [TestMethod]
   68         public void Can_SaveSomeUserDataViaStructureMap()
   69         {
   70             ContainerBootstrapper.BootstrapStructureMap();
   71 
   72             using (var uoW = ObjectFactory.GetInstance<IUnitOfWork>())
   73             {
   74                 var repo = ObjectFactory.GetInstance<IRepository>();
   75                 var newUser = new AccountUser
   76                                   {
   77                                       Name = "Test",
   78                                       Email = "Email Test",
   79                                       Password = "Password Test"
   80                                   };
   81                 var newUser2 = new AccountUser
   82                                    {
   83                                        Name = "Test2",
   84                                        Email = "Email Test",
   85                                        Password = "Password Test"
   86                                    };
   87                 repo.Add(newUser);
   88                 repo.Add(newUser2);
   89 
   90                 uoW.Commit();
   91 
   92                 //Clear the session so its forced to go to the database and not pull from the cache
   93                 uoW.CurrentSession.Clear();
   94 
   95                 var returnedAccountUser = repo.GetById<AccountUser>(newUser.Id);
   96                 Assert.AreEqual("Test", returnedAccountUser.Name, "Failed to get user from database.");
   97             }
   98         }

so this test was to tell me that I could write more than 1 thing to the database at the same time on the same UnitOfWork. I did use the nhibernate profiler to spy on a little of what was going on here, which may I add is superb, easy to use, understand and set up. Going back to the test though, the UnitOfWork and StructureMap did their job nicely and only created one of them. One UnitOfWork was created as part of the 'using' and the one that was injected into the Repository creation (on line 74 of the test) was the same one. Superb! The data was added, the UnitOfWork commited and job done. I cleared the session at this point to force nhibernate to fetch the data from the database again rather than retrieve it from its cache.

Test to prove that there was a different UnitOfWork created when created in another thread.
  110        [TestMethod]
  111         public void Must_HaveNewSessionPerUoWPerThread()
  112         {
  113             ContainerBootstrapper.BootstrapStructureMap();
  114             //Create this instance on this thread.
  115             var uoW = ObjectFactory.GetInstance<IUnitOfWork>();
  116             IUnitOfWork uoW2;
  117 
  118             var thread = new Thread(x =>
  119             {
  120                 uoW2 = ObjectFactory.GetInstance<IUnitOfWork>();
  121                 Assert.AreNotSame(uoW, uoW2);
  122                 Assert.AreNotSame(uoW.CurrentSession, uoW2.CurrentSession);
  123             });
  124             thread.Start();
  125         }
with this test I am thinking along the lines I could use the UnitOfWork per HttpRequest using MVC some where along the lines, creating the UnitofWork in ActionExecting maybe and the commit/rollback on what ever the opposite one is. When I get to working that out I will post that next I think.

Changes I would like to make for an MVC web application.
I will get round to bloging what I did for this, but for short I am considering removing the UnitOfWork and Repository all together and let StructureMap create the Session and inject it into the controller constructors when they are created, but my only hesitation in this is that it will couple NHibernate to the controllers as they will need an ISession injected into them and also need to manage the Transaction.