Tuesday 27 August 2013

OSX Guest account "Shakes" and won't log on after power failure

I have been borrowing a friends mac lately only for a couple of weeks while they are away. They have set up the guest account for me to log into. This is a great idea, it allows me to pretty much do what I like without any risk of damaging the computer (or so Apple will have you think). This assumption is almost correct, but as anyone with a little unix knowledge could tell you, "if you have physical access to the console and are able to reboot it, you can boot into single user mode". This means you can get root access :)

The problem is that this is a iMac, and there is a design flaw with it. The power cable goes into the monitor, this is the only power cable for the whole thing so if it comes out, your computer dies. Now Apple saw fit to put a highly reflective display on this thing so it needs constant adjustment to stop reflections, this means the power cable can work itself loose. Normally this might not be catastrophic since you could just plug back in and boot it up. Sure you might loose some work but since its an apple its unlikely your doing anything important. (Yes I am inferring that if you are a mac user you are probably just playing Facebook games or some sort or art, and yes thats not really important).

Now, rant over, if you are logged in as Guest when this power failure happens you will not be allowed to log back in. You will be faced with the login box, with the guest account, but instead of logging in you get an INFURIATING shaking box with no error message. So what does this mean? Who bloddy knows, the result though is you can't use the computer.

Where to now? Well Apple are no help here, so after a LOT of googleing I found a series of different blog posts that sort of related that I could string together into a solution. Here goes...

  1. Boot into single user mode: Reboot the mac and hold cmd+s you will end up at a terminal prompt as the root user... sort of. 
  2. Type:
    fsck -fy
    mount -uw /
    This will mount the root writable and give you root access to the machine.
  3. Type:
    passwd root  Enter a new root password.
  4. Type:
    dsenableroot -u root
    Enter the root password you just set three times and the root account will be enabled
  5. Type exit to boot up normally
  6. Log in as the root user by clicking other and entering root as the username and the password you set.
  7. Open System Preferences > Accounts  and disable the Guest account by deselecting both check boxes. Close the preferences window.
  8. This alone will not actually disable the account so open a terminal and enter:
    dscl . delete /Users/Guest
  9. Now go back to Accounts and re-enable the Guest user.
  10. Log out of the root account and you should be able to log in as guest now.
  11. OPTIONAL:



  12. If you don't want anyone to know you enabled the root account, you will want to disable it. Log on as root.
  13. Go back to Accounts > click Login Options
  14. Click the Join box next to Network Account Server
  15. Click Open Directory Utility
  16. Choose Edit(top menu) > Disable Root User

    So thanks Apple, that was a right pain in the arse. Fix this bug with the guest account please.

Sunday 25 August 2013

VMware consolidating disks fails with file lock error

Recently I came across an issue where I had a VM with lots of snapshot and delta disks but when you look in the snapshot manager there are no snapshots.

There is a nice little message in the vSphere client pointing to the issue.

When I attempted to consolidate the disks I got a horrible error, one of those VMware mystery errors.

"Unable to access file since its locked"

Super! Thanks VMware.

A bit of googleing around led me to a stack of articles talking about backup VM's having the disk mounted causing a file lock. I knew this was not the problem since this is a lab environment and there is no snapshot backup tool.

It did lead me to look on the array for file locks though. This is on a NetApp array so I ssh'd in and had a poke around.  I checked for locks on all of the files in the VM's directory and no matter what I entered I kept getting 'No Locks'


OK, so its not an array level file lock, what next?

I turned on SSH on a host that has access to the files, CD'd to the VMs directory to have a look around. The first thing I noticed that struck me as odd was 2 files starting with .lck-.

Since the VM was powered off these should not be there. So I rm'd both files and tried the consolidate again. HUZZAH!! its working.



Friday 2 August 2013

PowerShell one liners for Active Directory

Just like my PowerCLI one liners post, I am hoping that this one will improve and expand over time.

Get the user "Luke"
Get-ADUser -Filter {SamAccountName -Like 'Luke'}

Get all users with the Surname "Jones"
Get-ADUser -Filter {Surname -like 'Jones'}


Get all disabled users
Get-ADUser -Filter {Enabled -eq 'False'}

Enable a user account
Set-ADUser -Enabled $True -Identity "Luke"

Get members of a group
Get-ADGroupMember esxadmin | Select-Object SamAccountName, Name


Thursday 1 August 2013

PowerCLI one liners for VMware

Hopefully this will be a living post that I will update on a regular basis as I come across more little things I want to do.

Get a list of all VM names fast
Get-View -ViewType VirtualMachine -Property Name | Select Name

Get a list of all Host names fast
Get-View -ViewType HostSystem -Property Name | Select Name

Get a list of all VM's and their IP address (only works for powered on VM's)
Get-View -ViewType VirtualMachine -Property Name, Guest.IpAddress | Foreach-Object {Add-Member -InputObject $_ -MemberType NoteProperty -Name IpAddress -Value $($_.Guest.IpAddress) -Pa
ssThru} | Select Name, IpAddress


Get a list of all snapshots that are more than 2 days old and display its age in days
Get-Snapshot -VM $(Get-View -ViewType VirtualMachine -Property Name,Config.Template -Filter @{"Config.Template"="False"} | foreach { $_.name }) | Where-Object {$_.Created -lt $(Get-Date).AddDays(-2)} | ForEach-Object {Add-Member -MemberType NoteProperty -InputObject $_ -PassThru -Name Age -Value $((Get-Date) - ($_.Created)).Days} | Select-Object Name, Description, Created, Age


List any VM's that have CD drives attached (might stop vMotion working)
Get-Vm | Foreach-Object {$CD = Get-CdDrive -Vm $_; If ($CD.IsoPath -or $CD.HostDevice){$_ | Select-Object Name}}


Detach CD drives from all VM's
Get-Vm | Foreach-Object {$CD = Get-CdDrive -Vm $_; If ($CD.IsoPath -or $CD.HostDevice){$Null = Set-CdDrive -CD $CD -NoMedia -Confirm:$False}}


Turn on SSH for all hosts
Get-VMHost | Foreach-Object {  Start-VMHostService -Confirm:$False -HostService ($_ | Get-VMHostService | Where { $_.Key -eq "TSM-SSH"} )}

Turn off SSH for all hosts
Get-VMHost | Foreach-Object {  Stop-VMHostService -Confirm:$False -HostService ($_ | Get-VMHostService | Where { $_.Key -eq "TSM-SSH"} )}


Get a list of all VMs (including templates) with their vmx location, current host and folder and export to XML

get-view -viewtype virtualmachine -property name, config.files.vmpathname, parent, Runtime.Host | select name, @{n="vmxFilePath"; e={$_.config.files.vmpathname}}, parent, @{n="host"; e={$_.runtime.host}} | Export-Clixml -Path ./vms.xml


Import the list and add the VMs to inventory

Import-Clixml .\vms.xml | foreach { New-VM -VMFilePath $_.vmxfilepath -VMHost (Get-VIObjectByVIView $_.host.toString()) -Location (Get-VIObjectByVIView $_.parent.toString()) -RunAsync}


Get a list of all VMs and their configured OS 

Get-View -ViewType virtualmachine -property name, config.guestid, config.guestFullName | select name, @{N='guestid'; E={$_.config.guestid}}, @{N='guestFullName'; E={$_.config.guestFullName}} | sort guestid | Export-Csv -NoTypeInformation -Path ./vmguests.csv

Get report about host CPU, Network and Storage usage (real-time stats)
get-vmhost | foreach { 
$CPU = Get-Stat -Entity $_ -Stat cpu.usagemhz.average -Realtime | measure -Average -Maximum value
$net = get-stat -Entity $_ -Realtime -stat net.usage.average | where {$_.instance -eq ""} | measure -Average -Maximum value
$storage = get-stat -Entity $_  -stat storageAdapter.write.average -realtime | measure -average -maximum value
$_ | select @{N='Cluster';E={$_.parent.name}}, name, @{N='CPU MHz Usage Average';E={$CPU.average}}, @{N='CPU MHz Usage Average Max';E={$CPU.Maximum}}, @{N='Net KBps Usage Average';E={$Net.average}}, @{N='Net KBps Usage Average Max';E={$Net.Maximum}}, @{N='Storage KBps Usage Average';E={$CPU.average}}, @{N='Storage KBps Usage Average Max';E={$CPU.Maximum}}
} | Export-Csv -NoTypeInformation -Path ./Host_Usage.csv

Upload and download files over SSH

This is fairly easy to do if you are used to unix CLI and you are using unix/OSX. For windows as always there is a little more to it.

OSX/Unix

Open up a terminal. For mac users press cmd+space then type "terminal" hit enter, in Ubuntu press ctrl+alt+t, for any other unix I assume you know how to do this.

Navigate to the file that you want to upload or the location that you want to download the file to using the cd command. eg. cd ~ (this will take you to your home folder)

Now you need to use the scp command to copy your files. it is used like this:
scp <file to copy> <destination> 

If you are using a non standard port (not port 22) for SSH they you will need to specify the port with the -P flag. If you want to copy a folder then you need to specify the -r (recursive copy) flag. 
scp -P 443 -r <file to copy> <destination> 

Your remote file path needs to specify your username, server and file path. For example if I wanted to copy example.txt from my home folder to my home folder on my SSH server I would type:
scp -P 443 /home/luke/example.txt  luke@icitd.com:/home/luke/
If I wanted to copy the same file from my SSH server to my local machine I could type:
scp -P 443 luke@icitd.com:/home/luke/example.txt  /home/luke/txt/

If I wanted to copy the whole txt folder from my home directory to my SSH server I would type:
scp -P 443 -r /home/luke/txt  luke@icitd.com:/home/luke/
remember the -r to recurse the directory!

There are GUI based tools available for OSX and linux if you like, examples are:
Ubuntu - Search the app store, or Filezilla is available

Large files

If you want to copy a large file, a lot of files, or you regularly want to back up a directory then rsync is the tool for you. It is used in a similar way to scp but it only copies the changes to files so if you have a large amount of data that doesn't change much this can save you a lot of time.

Windows

WinSCP is a GUI file transfer program for windows, it has a midnight commander type interface or a standard interface. I find the midnight commander interface to be much more useful.

Download and install WinSCP, open WinSCP.

You can use either SFTP or SCP for the file protocol, I have not noticed any major differences but SFTP is supposed to be the better protocol. 
In the Host name box enter the IP or hostname of your SSH server, enter the port number you are using (default is 22)
Enter your user name and password, click Login.

Agree to saving the fingerprint when prompted, this will open up the file navigator. From here you can drag and drop files from your client to server or server to client. Your local machines files are on the left and the remote machine is on the right.

Socks proxy over SSH OR Safely browse the Internet

This one comes in handy for me in three different scenarios. The first is when I am travelling and relying on free internet hotspots (eg. airports, Mc Donalds, hotels) and I want to safely do banking. Another is when I want to look at a website that is blocked by the network I am attached to. Finally, so that I can look like I am at home when I am not to a website that uses location data based on an IP address.


What you will NEED

  1. A working SSH server that you have remote access to. If you don't then have a look here.
  2. Firefox (you can do this without firefox if you install third party add-ons or are not using windows)

How to do it

Mac

Open a SSH session and create a dynamic port forward using the -D flag, you can also add the -C flag to compress the session, this probably wont help much since most web servers already compress the response. eg. ssh luke@icitd.com -C -D 6666

Windows

Open Putty, enter your server details eg. ssh luke@icitd.com
Click SSH, Check 'Enable compression' 

Expand SSH and click Tunnels
Enter 6666 in the source port and select 'Dynamic'
Click Add
(Optional) Click back on 'Session', enter a name for the session and click save

Click Open, Click Yes to accept the key if this is the first time you are connecting
Enter your password (there will be no feedback when typing the password)
You should get a black screen that says <username>@<hostmane>:~$ 

Install Firefox, start it up then go to the settings page. These example screen shots are for windows but the process is very similar for mac or linux.

  1. Click on the firefox menu on the top left, click options, click options
  2. This will open the Options dialogue box
    click the advanced icon at the top, click the network tab below the icons, click the settings box under Connection.
  3. In the Connection settings box click Manual proxy Configuration, in the SOCKS host box type localhost, in the port box beside type a high order port (eg. 6666), click OK to close the Connections settings box, then again to close the options box.

  4. OPTIONAL: Firefox will still do the DNS lookups on your local network even though it then tunnels the web page over the SSH session. If you are paranoid and don't want your DNS lookups known then you can tell Firefox to do remote DNS lookups.
    1. In Firefox, enter about:config in the url bar, click I'll be careful...

    2. In the search bar type socks, double click network.proxy.socks_remote_dns to change the value to true.
      here
    3. Now Firefox will do remote DNS queries.
Thats all there is to it, now all your web traffic is encrypted between you and your home network. To external sites the traffic will seem to come from your home IP address. 
Happy browsing!

Mount NTFS as read/write in OSX Mountain Lion

I hate Apple.  Okay, I don't wholly hate them, but I hate how otherwise simple tasks in Linux or Windows become pains in the neck on OSX.  I also hate how Apple deliberately and unabashedly devote their development and UI design to ensure that you, the consumer, are locked into the Apple ecosystem forever and ever, amen.

Case in point? Something as simple as mounting NTFS devices as read/write.  Apple should be absolutely and utterly ashamed that OSX didn't support this natively until only recently (OSX Mountain Lion), as basically any other respectable OS (Linux, BSD, Unix, Windows) could do so back in the year 2000.  How Apple made it this far without being publicly humiliated for this simple lack of functionality, I don't know.  But the good news is although its not supported via the GUI, we can now do so via a simple mount command.

First, eject the NTFS disk(s) that appear in finder by clicking the eject button next to the partition/disk:



Then open a terminal windows and run the following command:

sudo mount -t ntfs -o force,rw,nobrowse /dev/disk3s2 /media

Where /dev/disk3s2 is the disk partition you're trying to mount and /media is the mount location.  I just created /media using sudo mkdir /media since this directory does not exist inherently on OSX.  Note that the force command and nobrowse commands are the key differentiators to mounting an NTFS disk in Linux.



And then you're laughing.  A super simple command, but it can save hours of headache.

More to come later for OSX tips to allow you to actually use your Apple computer as a computer, and not as an Apple revenue generation machine.

RDP over SSH

If you want to remotely access your desktop when you are away from home then read on...

While it is possible to just use your router to forward port 3389 to your desktop and then open an RDP session from anywhere, there are some distinct advantages to doing this over an SSH session.

Compression

The first reason is the ability to compress the data stream. This will add some CPU overhead to both your client and server to compress and then decompress the data. In my experience this extra CPU usage is tiny, even when using a Raspberry PI the CPU can easily handle it. The effect on the usability of the RDP session, on the other hand, is massive. This all depends on the upload speed from your network but an uncompressed session is generally pretty unusable, where a compressed session is comparable to a local session over a LAN.
When connecting your SSH session you can specify the -C flag. This flag tells your client and server to compress the data that is sent over the network. This will reduce the amount of data flowing out from your home network (a plus if this is counted towards your monthly data allowance) by compressing the data.

Security

The RDP protocol was never designed to be Internet facing, if you google it you will find a number of ways to brute-force or dictionary attack the password to gain access. By using SSH you can implements key based authentication to ensure far better security.

So how do I do it?

I am assuming that you have a working SSH server and have remote access to this server. If you don't then have a look here. You can also install cygwin on your desktop to have an all-in-one solution.

When you establish your SSH session you need to tell it to do 2 things; compress the session and create a port forward for the RDP session.
You specify compression with the -C flag on the command line and in PuTTY click on SSH on the left, then check "Enable Compression".

The next thing to do is set up the port forward. This works by creating a link between 2 ports one on your client to another on your server. This means any traffic that goes to the local port will travel to your SSH server over the SSH tunnel, then the server will forward it to the destination on your LAN.

Unix/Mac

The command looks like this: -L 5566:192.168.0.5:3389. Here is what each part of the command means:

-L - Local port forward, we are forwarding a local port on our client to the server.
5556 - The local port to be forwarded is 5556 (in this example)
192.168.0.5 - The IP address of your desktop on your LAN (you could use a hostname if your SSH server can resolve it)
3389 - The remote port that the traffic will exit on

Windows/PuTTY

In PuTTY you specify the same details in a slightly different way. Expand "SSH" in the left panel, then click "Tunnels". Enter your local port 5566 into the "Source Port" box, then enter 192.168.0.5:3389 into the "Destination" box. Don't forget to click "Add"!

Connecting your RDP sesison

Now the easy bit, open up the Remote Desktop client and connect to localhost:5556. This will tell the client to connect to your local machine on port 5556, since we have forwarded that port to port 3389 on our home computer this will create a connection home.

 Enter your user name and password and you will be looking at your home desktop!


How to access your home network from anywhere safely

I have been asked by friends and colleagues how to do this on a number of occasions so I thought I would put a step-by-step guide together.

Why would you want to do this??

There are a number of reasons that I do this on a regular basis. Here are some:
  1. RDP to my Windows VM at home
  2. Browse the internet safely from an unsafe location (eg. airport, internet cafe)
  3. Look for files on my NAS
  4. Download or upload files to my NAS
  5. Start a download whilst I am not home so it is finished when I get home
  6. Remote access to my home lab environment
  7. Circumvent the security of the network I am currently connected to (eg. Facebook/ebay/etc. at work)
If this sounds like something you need to do then read on...

Things you will NEED

  1. SSH server (physical computer, old laptop/netbook, apple tv, NAS, old android phone)
  2. Router that is configured to forward SSH traffic to your server
  3. Static IP address OR a DDNS hostname (something like dyn.com/dns or www.noip.com. There are heaps of options)
If you already have these things set up then go on to How to use

Setup

Set up your SSH server. The example I will give here is for Ubuntu but should apply to most distros.
  1. Install Ubuntu (I wont cover this since Ubuntu do an excellent job of that themselves here)
  2. Set a static IP address for your server on your LAN eg. 192.168.0.x
  3. Install OpenSSH Server (Again Ubuntu doco is excellent)
  4. Forward traffic coming from the internet on port 22(SSH port) to your the internal IP address you previously assigned to your SSH server.
    • The process to do this is different for all routers, have a look at your routers doco or google it
  5. OPTIONAL: A lot of networks will block traffic leaving on port 22, this will mean that you will not be able to SSH to your home server if you are listening on the standard port. The solution to this is to make your SSH server listen for connections of port 443(SSL) I haven't found any networks yet that have 443 blocked. Read below for instructions

Changing the port your server listens on

There are 2 ways to do this:
  • If your router can forward incoming port 443 to your SSH server on port 22, you're in luck. Just set that up and all external traffic on port 443 will be passed to your server on port 22.

  • If your router can't do this port translation then you need to configure it to pass the traffic on 443 to your SSH server, then make your server listen on port 443.
    1. Make a backup of your configuration file using these commands
      sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.original
      sudo chmod a-w /etc/ssh/sshd_config.original
    2. Edit your sshd_config file, find the line Port 22, make sure it is uncommented and change the 22 to 443 (you can listen on multiple ports by having multiple Port commands in the config file) I use nano to edit files from the command line, but you could use vim, just remember to sudo.
      sudo nano /etc/ssh/sshd_config
    3. Save the file by pressing ctrl-x, y, enter
    4. Then restart your ssh server. (This won't kill any active SSH sessions)
      sudo service ssh restart
    5. Test that your SSH server is working by typing ssh localhost -p 443. This should prompt you for your password then drop you at a bash prompt. Press crtl-d to exit the session.

How to use this wonderful new thing

From an external network do the following:

Unix - including Mac

If you have an unix based OS then you probably already have all the tools you need to connect to your server. 
  • Open up a terminal (on a mac press CMD+Space, then type terminal, hit enter)
  • Enter into your terminal ssh <your user name>@<your IP or hostname> -p port
    • For example: ssh luke@icitd.com -p 443
    • The -p tells your ssh client to use port 443 for the connection (if you are using port 22 then you can leave this out)
  • The first time you connect you will get asked if you want to add the servers fingerprint to your local known hosts file. Type yes and press enter
  • Enter in your password (Note. no characters will show when you are typing your password, not ever *'s)
  •  .. and you should be connected to your home server :) Give yourself a pat on the back.

Windows

  • Download and install PuTTY - PuTTY download page - Direct link to the installer
  • In the hostname box enter <your user name>@<your IP or hostname>. eg. luke@icind.com
  • In the port box enter the port number you are using (22 is default, 443 if you're following my guide)
  • Click open
  • Enter your password when prompted (Note. no characters will show when you are typing your password, not ever *'s)
  • You should now be connected to your home SSH server :) Give yourself a pat on the back.

Android

  • Open the play store and install ConnectBot
  • Open ConnectBot
  • In the username@hostname:port box enter <your user name>@<your IP or hostname>:<port>, eg. luke@icitd.com:443 (the port can be left off if you are using port 22)
  • Tap Done, enter your password when prompted.
  • You should now be connected to your home SSH server :) Give yourself a pat on the back.

iPhone

  • Your outa luck here. I don't have nor have ever had an iPhone so I can't help you. Your good mate google can probably help out though ;)

Now What?? How do I do all the cool stuff you listed above?

Read the following articles, thats how!

Socks proxy over SSH (safely browse on unsafe connection)
Remote torrent - coming soon
Get around IT/network security - see Socks Proxy.

Security

Coming soon