Wednesday 28 May 2014

Git: Just Getting Started, Barely Scratching the Surface

Scene: In my (precious-little) spare time, I'm working on Project Euler. I haven't done any proper mathematics since I left Uni >10 years ago (when_where I completed a Major in Pure Mathematics), so it's all feeling more than a little rusty. I'm also teaching myself Python and again working on projects bigger than 10-line infrastructure scripts, so it's a good [re-]learning experience.

So, 10 Euler Problems in, I'm realising that certain patterns recur in the Problems, and it will make my life *a lot* easier if I use modules & descriptive functions names, and create reusable code. I am also realising that I strongly need source code control - I've only ever really used Subversion (which I did quite like, but never really used many advanced features), but why not throw another log on my bonfire of learning? So additionally learning Git it is.

Topics covered:

I plan to cover off:
  • Creating a new Git master/shared repo based on an existing unmanaged set of files [source code] from a client machine
  • Create some cloned repos
  • Basic file checkins.
Things I know are missing from this post:
  • Branching in Git
  • Multi-master push/pull stuff (which Git was truly built for)
  • Updating your local client's repo from the main shared repo which someone else has updated (which I think is called "rebasing" in Git-land)
  • Anything else not-completely-basic
Assumed:
  • All steps below assume use of Linux/*nix - please adjust Windows/Other commands as you require
  • Access from your client machines' user accounts to a shared host via ssh (ssh access from client->server)
As per previous blog entries, this is less about education on the topic as a whole, and more about recording the steps so that I can do it again. This probably deserves an extra warning: you may well not learn anything in this post that isn't better documented elsewhere. Also, I'm not really a Developer, so again, there are probably better examples to follow elsewhere. That said, I haven't seen this particular steps/combo in my explorations, so I feel the need to record it.

Quick note: I'm using Git a bit like Subversion (ie single-master)

I know Git is fully decentralised source control, and I'm not using it in the best/most idiomatic way, but as an opening gambit I need to cocoon myself in the ideological constructs that I know already. So, we're going to construct a Git repo system setup in the way I used to use Subversion. The following steps will take a create a shared repo on a shared-access host that you can think of as a [master] subversion repo, and then use this repo as the central source of checkins and checkouts.


Step 0: Configure Git on all hosts (Shared and Client)

Make sure git is installed on all systems (run as root):
if [ `which yum` ] ; yum install -y git; fi
if [ `which apt-get` ] ; apt-get install -y git; fi

Configure your user account(s):
git config --global user.email "email@email.com"git config --global user.name "My Name"

(I'm sure I'm missing other useful user info here, but the above info is minimally required for later commits).

Bonus extra step: configure user-wide git file excludes (ie: tell git to permanently ignore certain files/types/directories when assessing the checkin status of a working directory). This helps to ignore temporary/overwritten/cache files.

# Create a global "ignore" file in your homedir. This is minimal file based on what has annoyed me so far - see references below for better ideas
cat ~/.gitignore_global > <<EOF
# Python Byte-compiled / Optimised / DLL files
__pycache__/
*.py[cod]

EOF

git config --global core.excludesfile ~/.gitignore_global
References:
Guide on creating Ignore files : https://help.github.com/articles/ignoring-files
A lot of pre-configured Ignore files: https://github.com/github/gitignore

Step 1: Create a "Master" Shared Repo

Create a Shared Repo on a shared server ("SharedHost01") that you have ssh access to. I have run this command on SharedHost01 itself - there may be other ways to do this.

cd ~
mkdir projectName.git
cd projectName.git
git init --bare

Step 2: Create a Client Repo (from Existing Content)

This step can probably be done better - eg skip the copy step and clone straight from master over the top of existing - I have no idea of the semantics of overwrite behaviour, I was just being overly cautious because I didn't want to lose my existing code!

This step is run on your client machine ("ClientHost01"), that has existing content you suddenly realise that you need to manage.
 Assumption:  existing project is in directory "~/projectName", so parent dir is ~.

# Create a backup of the original code in case we get this wrong
cd ~
mv projectName{,.orig}
mkdir projectName
cd projectName
# Now we pull down a blank project from Shared Host
git clone user@shareHost:projectName/
# Copy in the existing code
cp -rp ../projectName.orig/* .
# Add all content (aside from the globally-ignored files from Step 0...)
git add .
# Check it into you local on-machine repo
git commit -m "Initial ProjectName checkin, with existing codebase"
# Now push this "new" code back to the Shared Host
git push origin master

Step 3: Set Up a Second Client Repo

cd ~
mkdir projectNamecd projectName
# Now we pull down a the initial code we just checked in
git clone user@shareHost:projectName/

And that's it for now - the baove has pretty much worked for me so far, so here's hoping I haven't got it horribly wrong. :)

Other References:

I have used several pages from this guide: https://www.atlassian.com/git/tutorial/git-basics

Monday 12 May 2014

Configuring a new Windows Machine


Today I learnt a new way to configure a new Windows machine: with a PowerShell package manager. A bit like like YUM for Windows! All thanks to https://chocolatey.org/. As a side-note, this package manager, although currently third-party, will be built into Windows itself fairly soon, via One-Get in Powershell v5.

This assumes you have Powershell >= 3 installed already.

Into a cmd Window:
@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%systemdrive%\chocolatey\bin

And into a PowerShell Window, some useful stuff:
cinst notepadplusplus.install
cinst Firefox
cinst GoogleChrome
cinst 7zip
cinst flashplayeractivex
cinst flashplayerplugin
cinst putty
cinst sysinternals
cinst 
cinst procexp

cinst sublimetext2
cinst curl
cinst Wget
cinst winmerge
cinst wireshark



Remove Windows 8 Default CrApps:

And a bonus Windows 8 tidbit that I made up on my very own: how to remove a lot of crappy apps shipped by default with Windows 8.1 (and maybe in 8.0) in PowerShell:

$CrappyApps = @("Microsoft.BingFinance ","Microsoft.BingFoodAndDrink",`
"Microsoft.BingHealthAndFitness","Microsoft.BingMaps",`
"Microsoft.BingNews","Microsoft.BingSports","Microsoft.BingTravel",`
"Microsoft.BingWeather","Microsoft.HelpAndTips",`
"microsoft.windowscommunicationsapps","Microsoft.WindowsReadingList",`
"Microsoft.XboxLIVEGames","Microsoft.ZuneMusic","Microsoft.ZuneVideo",`
"CheckPoint.VPN","f5.vpn.client","JuniperNetworks.JunosPulseVpn","Microsoft.MoCamera","SonicWALL.MobileConnect")
 

$CrappyApps | % { Get-AppxPackage -Name $_ | Remove-AppxPackage 2>&1 | Out-Null }

Note, the above few commands haven't been run in production - I ran each Get|Remove command individually on my own machine, and then arrayified it all for this post, so there may be an error lurking in there.

Wednesday 7 May 2014

Modifying Web Server SSL settings to current modern web standards


This is not a post talking about the whys and wherefores of the settings below; nor is it mean to be a discussion of SSL security in general - just a quick reference guide to setting SSL responses in different web servers & OSes. This post will be edited as I find the need to configure different servers and software.

Apache on Linux:

Edit /etc/httpd/conf.d/ssl.conf (on RHEL6; other OSes move this file around).

Comment out the current SSL settings:
#SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
#SSLProtocol all -SSLv2


Add settings:
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AES:RSA+3DES:!ADH:!AECDH:!MD5:!DSS
SSLHonorCipherOrder     on

# This next one cuts out IE6 on WinXP (good riddance)
SSLProtocol all -SSLv2 -SSLv3



Note/update, Oct 2014: The advice in this post predates the POODLE SSL attack, but already disabled SSLv3 anyway, so the recommendations have not needed updating in light of recent SSL attacks - it is still fully valid advice.

NTP on Cisco IOS


To round out my NTP notes, here is NTP on Cisco IOS.
Timezone is  Adelaide time, GMT+930 (Australian Central Standard, ie ACST).
Remote NTP server is Internode's - substitute for (or add to) your preferred ISP/pool.ntp server.

! Set timezone
clock timezone ACST 9 30
clock summer-time ACDT recurring 1 Sun Oct 2:00 1 Sun Apr 3:00

!Set NTP for local NTP server
ntp server 192.168.1.254
!Set NTP for remote NTP server

ntp server ntp.on.net

!Does this router have a DNS server set? If not, add it for NTP resolution
!ip name-server 192.168.1.100



Tuesday 6 May 2014

NTP on Linux


A Linux host needs NTP set to ensure correct time sync. These commands below set this for Linux systems, with an emphasis on Australian settings (swap in other NTP servers for non-Australian servers). This can be set simply by pasting the commands below into a Bash prompt (as root).

This guide makes no attempt to check/enforce the security of the NTP server: issues such disabling commands such as "mon" are not covered here, and appropriate firewalling is assumed. The NTP config file contained in Red Hat Enterprise Linux has a secure-by-default config (in RHEL6, if not prior as well), and the commands below simply assume security and configure the time sources.

All commands are IPv6-compatible, although only IPv4 is used in the addressing below.

Please note that Red Hat Enterprise Linux 7 has introduced Chrony as the default NTP service instead of the venerable NTPd - see notes at the end for chrony config.

Systems with NTPd

Install NTP on the system, strip defined servers

# For yum/RHEL-based systems
if [ `which yum` ] ; then yum install -y ntp; fi
# For apt/Debian-based systems
if [ `which apt-get` ] ; then apt-get install -y ntp; fi
# Backup original config
cp -p /etc/ntp.conf{,.orig}
# Strip all default servers
perl -i -pe 's/^server/#server/' /etc/ntp.conf


# Optional: Configure local timezone
ln -sf /usr/share/zoneinfo/Australia/Adelaide /etc/localtime

Add Local Servers

cat >> /etc/ntp.conf <<EOF
# NTP servers
server 192.168.1.1 prefer # Set this to your local NTP-serving machine if you have one
server 3.au.pool.ntp.org


EOF


Add Internode or Telstra Servers

Only required if you are on Internode networks:

cat >> /etc/ntp.conf <<EOF

# Internode NTP server

server ntp.on.net

EOF


Only required if you are on Telstra networks:

cat >> /etc/ntp.conf <<EOF

# NTP servers
server tic.ntp.telstra.net
server toc.ntp.telstra.net

EOF

Optional Step: Allow Local subnets to query this NTP

This will allow other machines on your local network to query this NTP server. Remember to allow inbound port UDP:123 on your host firewall.

cat >> /etc/ntp.conf <<EOF
# Allow Local subnets to query this NTP
restrict 192.168.0.0 mask 255.255.0.0 nomodify notrap

EOF



Ensure NTP is started & starts on boot

Commands tested on RHEL6 only; other OSes left as an exercise for the reader.

service ntpd start
chkconfig ntpd on
 
 

Systems with Chrony (RHEL7 and others)

Red Hat Enterprise Linux 7 uses Chrony as the default NTP daemon - unless you have a good reason to use ntpd, then you can simply configure chrony the same way as above.

yum install -y chrony
# Show config
timedatectl
# Set timezone
timedatectl list-timezones | grep Adelaide
timedatectl set-timezone Australia/Adelaide
# Show NTP status
chronyc sources
# Change NTP config
perl -i -pe 's/^server/#server/' /etc/chrony.conf
cat >> /etc/chrony.conf <<EOF
# NTP servers
server 192.168.1.1 iburst # Set this to your local NTP-serving machine if you have one
server 3.au.pool.ntp.org iburst
EOF
chronyc sourcessystemctl restart chronyd.servicesystemctl enable chronyd.service


NTP on Windows


A Windows Domain needs NTP set on it's PDC Emulator to ensure the domain has correct time sync. These commands set this for Windows systems, with an emphasis on Australian settings.


This can be set simply by pasting the commands into a cmd window (I recommend setting NTP1 to your actual local NTP server/router if you have one). These commands are expected to work on Server 2008 onwards.

SET TARGET=localhost
SET NTP1=192.168.1.1
SET NTP2=2.pool.ntp.org
SET NTP3=3.pool.ntp.org
w32tm /config /computer:%TARGET% /update /manualpeerlist:"%NTP1% %NTP2% %NTP3%" /syncfromflags:MANUAL


Australian Settings

Same commands as above, just swap in these variables instead.

Internode version:

SET TARGET=localhost
SET NTP1=192.168.1.1
SET NTP2=ntp.on.net
SET NTP3=3.au.pool.ntp.org


Telstra version:

SET TARGET=localhost
SET NTP1=192.168.1.1
SET NTP2=toc.ntp.telstra.net
SET NTP3=tic.ntp.telstra.net

Confirm Settings

Validate this with the query command:
w32tm /query /configuration /computer:%TARGET%

Outlook: Migrate Outlook Profiles to Office 365 (Wave 15)


Office 365, the current Exchange-2013-based incarnation ("Wave 15"), has (semi-)easy migration options from on-premises... except for the client-side, if your client-side is an existing Outlook 2007/10/13 based setup. Third-party tools, such as MigrationWiz, can fill the gap (possibly quite well), but if these are not an option, then read on.

Wipe Outlook Profiles & Start  Again

Now, the fastest way forward to migrate a fleet is to simply wipe all Outlook profiles & start again. Not that nice, but as long as your Autodiscover is working (and it is, isn't it? :) then all a migrated user has to do is click "Next" a lot when next opening Outlook.

Current advice that I found around the web is usually to just delete this reg key:
HKCU\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem\Profiles
but this results in a horrible, confusing prompt when next opening Outlook ("select profile") which helps noone migrate. So don't do this. :)


My advice is to create a Group Policy, which uses GPO Preferences to delete several more reg keys/values and also a Folder. Set each entry in the Preferences to only Apply Once, and to Run in Users Security Context. All of the settings are User Settings, and not Machine.

Make your GPO delete the following reg keys:
HKCU\SOFTWARE\Microsoft\Office\14.0\Outlook\Setup\First-RunHKCU\SOFTWARE\Microsoft\Office\14.0\Outlook\Setup\FirstRun
HKCU\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem\Profiles
And to also delete the folder:%LOCALAPPDATA%\Microsoft\Outlook

Replace/add more keys for the value above of 14.0 with as many versions  of Outlook as you wish to cater for [14.0 = Outlook 2010, 15.0 = 2013, etc].

Then:
  • Set your GP's permission to only the users you wish to migrate (ie remove the Authenticated Users from the GPO; then for testing, make it only one test user; for Staged, simply add the each set of users as you stage-migrate them; for Cut-over, make it all users)
  • Set the Computer Settings of this Policy to Disabled (all above setttings are User)
  • Link the Policy to any/all OUs with User Accounts.

Can't I just use a PRF File with a Server Name?

Gone are the days (Wave 14 and prior) where you could simply apply a PRF file or other such niceities - Microsoft now hides the server name field and uses a per-user GUID (assigned when first migrating a user) as a Virtual Connection-Point. If your aim is to fully seamelessly migrate Outlook profiles (without  third-party tools) utilising manual configuration, then I leave this as an excercise to the reader (hint: you will need to create a per-user PRF file which sets up the VCP and uses the HTTP proxying connection attributes, and then apply this PRF to each respective user)




The Start

Hello and welcome,

This blog is primarily dedicated to small notes on IT stuff as I find it - in 15+ years in IT, I've found, used and implemented more tidbits than I could possibly remember. I have often found myself needing to refer to them 10+ years later, and wished I had recorded the more arcane bits earlier, rather than re-finding them again. So, I plan to simply record small items as I go, with  the aim of being a useful reference for myself and others in years to come.

Milton.