avjaarsveld to #{1.0/0.0} and beyond!

Development Workflow Process

Some examples of Development Workflows

My current Development Workflow (using Jira, Bitbucket, Git, AWS)

This process works well as part of a larger development team

*---------------------------------------*
| Select first open ticket from backlog | <--------------------------.
*---------------------------------------*                            |
                  ||                                                 |
                  \/                                                 |
*-----------------------------------*                                |
| Set Jira ticket to In Development | => *-----------------------*   |
*-----------------------------------*    | Branch from master,   |   |
                          *---------*    | the name should start |   |
     .------------------->|         |    | with Jira ticket name |   |
     |   .-----------.    | Do work | <= *-----------------------*   |
    (no)-| finished? | <= *---------*                                |
         .-----------.                                               |
               |      *-----------------------------------------*    |
             (yes)--->| Create pull request (to merge into dev) |    |
                      *-----------------------------------------*    |
                        ||    _____________________________________  |
                        L==> ( Ticket moved to Review lane in Jira ) |
                              -------------------------------------  |
           .---------------------------------.     ||                |
           |                                 | <===                  |
     (yes)-| Pull request to review present? |                       |
       |   .---------------------------------.-(no)-------------------
       v
*-------------*    .-------------.        .-----------.--------(yes)
| Select pull | => | has issues? |-(yes)->| critical? |          |
| request     |    .-------------.        .-----------.---(no)   |
*-------------*          /    ^       ______________________|     |
                        /     |      |                           v
*-----------------*    /      |      v       *-----------------------*
| Approve request |<-(no)  *--------------*  | Reject pull request - |
*-----------------*        | Add Comments |  | New request required  |
        ||                 *--------------*  *-----------------------*
        \/
 ___________________________
( Ticket moved to test lane )
 ---------------------------

My previous Development Workflow (using Redmine, Github, Git, AWS)

This process works well as part of a smaller development team. There is more testing to compensate for fewer reviewers and the lack of a design, QA and dev-ops team. The developer does more of the testing and dev ops work

  1. Create a branch from the staging branch

    The name should be or start with the Redmine ticket number

  2. Do some work

    Make some changes, commit them, push them (to Github)

  3. Create a pull request to merge your branch into the staging branch and ask someone else to review the code before merging it.

    If issues are raised, action them and update the pull request

    If there are no issues, merge the branch and test on the AWS Opsworks staging stack

  4. Ask anyone you can find to test functionality on staging

  5. Merge your branch with the per-production branch and test on the AWS Opsworks pre-production stack

    pre-production testing differs from staging testing in that real connections to payment gateways and order processors are used

  6. Create a pull request to merge your branch into the master branch

  7. Finally double-check the new functionality on live


Read more...

Moving From Sublime Text 2 To Text 3

I’ve recently “upgraded” from Sublime Text 2 to Sublime Text 3. Here are some notes about how to get Text 3 setup for better productivity and to keep my Colour Theme.

Installing Add-Ons (Packages)

E.g:

  1. CMD + Shift + P
  2. Package Control: Install Package
  3. GitGutter

Packages installed: GitGutter, BetterCOffeescript, SublimeLinter, SidebarEnhancements, AllAutocomplete, BracketHighlighter

For SublimeLinter: CMD + P Toggle Linter to enable/disable linters.

Copying Colour Theme

This is how I moved my colour theme from Sublime Text 2 to Text 3. It works, but is probably not the best way to do it. If you know of a better way, please leave a comment.

cd /Applications/Sublime\ Text\ 3.app/Contents/MacOS/Packages/
cp Color\ Scheme\ -\ Default.sublime-package ~/Documents/
cd ~/Documents/
unzip Color\ Scheme\ -\ Default.sublime-package -d temp
cd /Users/me_username/Library/Application\ Support/Sublime\ Text\ 2/Packages/Color\ Scheme\ -\ Default/

Make your edits. In my case I copied over my Monokai Faded colour theme file as shown above.

cp Monokai\ Faded.tmTheme ~/Documents/temp
cd ~/Documents/temp

Compress all files in folder to a new archive called Color Scheme - Default.sublime-package

cp ~/Documents/temp/Color\ Scheme\ -\ Default.sublime-package /Applications/Sublime\ Text\ 3.app/Contents/MacOS/Packages/

Read more...

Connect to an AWS instance from Ubuntu and copy the Postgres Database

Connect to the instance, using the terminal

ssh ubuntu@public_ip

Given the instance’s public_ip (e.g. 51.82.211.212), which you can find listed in the AWS console. At this point your public key should already be associated with your AWS account, you can find more details here: help.github.com/articles/generating-ssh-keys/

Use pg_dump to create a copy of the Postgres database

pg_dump -Fc -h DB_Host -U root name > db.dump

DB_Host is listed in the RDS App detail’s under “Environment Variables” in the OpsWorks console. It should look something like name.c2bahyyit4qt.sa-north-1.rds.amazonaws.com

Enter the password

my_password

Copy the file from the remote instance to local computer

On the local computer:

scp ubuntu@public_ip:db.dump ~/

Import db dump into local db

bundle exec rake db:drop && rake db:create && pg_restore --verbose --clean --no-acl --no-owner -h localhost -U username -d db_name /home/me/db.dump

Use pg_restore --verbose --clean --no-acl --no-owner -h localhost -U username -d db_name /home/albert/db.dump if restoring to a database that has just been created (drop and create not needed).

Other Tasks - e.g. Launching the Console

Switch to deply user

sudo su deploy

Go to the app path

/srv/www/name/current

Launch Console in Production Mode

bundle exec rails console -e production

Read more...

Troubleshooting Sidekiq

Starting Rake Sidekiq

Connect to AWS instance

Then…

sudo su deploy
cd tmp/pids/
ll
rm sidekiq.pid 
cd ..
..
bundle exec rake sidekiq:start
cd ..
cd log
ll
exit

Checking Sidekiq Log (while running a job)

sudo su deploy
cd /srv/www/my/current
tail -f log/sidekiq.log

Sidekiq won’t stop or start

Try stop:

su su deploy
cd /srv/www/me/current
bundle exex rake sidekiq:stop
``

Try start:

su su deploy cd /srv/www/me/current bundle exex rake sidekiq:start ``

If those fail, try delete the sidekiq.pid and try start again:

rm -rf tmp/pids/sidekiq.pid 
bundle exec rake sidekiq:start

Read more...

Setting up a RVM, Ruby, Rails, Bundle, Git on a Macbook

Not tested yet - may be incomplete

I’ve got another new laptop. This time it’s a Macbook Pro, so I thought I’d document what I do to set up my Mac development environment, to make it easier for me next time, and to help you out…

I created these notes while doing the setup, but I have not yet had a chance to review or test them (I’ll do that the next time I set up a Mac development computer). In the mean time, please excuse any errors or omissions (or better yet, let me know in the comments)

1. Installing the basics

The OS was already installed, along with the usual Mac goodies. You may need to install git (see below).

3. SSH Keys are required to connect to github

This is to-do.

5. Install RVM (and Rails etc)

If, like me, you have multiple projects with different gems, using different version of Rails, you’ll need RVM. I installed Rails via RVM (after installing RVM) and installed MySQL separately. This is how:

Note: Follow the latests instructions on railsapps.github.io for Mac OS X version 10.10 or moncefbelyamani.com for Mac OS X version 10.9 (and older) and/or rvm.io/rvm/install

If you have Ruby already installed, you do not need to remove it, RVM will leave it untouched.

1. Install Xcode Command Line Tools

First check that it is not already installed. xcode-select -p should return /Library/Developer/CommandLineTools and gcc --version should not return an error message.

If not installed, install using:

xcode-select --install

2. Setup git

Git is automatically installed as part of the Xcode Command Line Tools (see above).

Check if git has already been set up using git config -l --global.

if nessesary, add your username and email for git:

git config --global user.name "Peter Pan"
git config --global user.email "peter@neverland.com"

4. Clone a repo into a new folder

git clone git@github.com:pirate_bay/pirate_ship.github.io.git pirate_ship
git clone https://avjaarsveld@bitbucket.org/neverland_resort/atm_system.git atm_system

1. Install Homebrew

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Check that Homebrew is installed/setup correctly using brew doctor which should return Your system is ready to brew.

5. Intall RVM

\curl -sSL https://get.rvm.io | bash -s stable --ruby

Run source /Users/me/.rvm/scripts/rvm in all your open shell windows as instructed

Test RVM with type rvm | head -1 which should return rvm is a function

Optionally Install a version of Ruby

rvm install 1.9.3

The above installs a specific version. rvm install 1.8.7-p374 installs a specific version and patchlevel. rvm install ruby --latest installs the latest version.

Create a Gemset for installed Ruby version

List gemsets with rvm gemset list

This is also to-do.

Install Bundler

Bundler is installed with Ruby or RVM

Used bundler to install Rails and other gems using bundle install

Install gems listed in Gemfile:

bundle install

This failed as capybara-webkit failed to install. I fixed this by instaling Qt first brew install qt.

bundle install failed again as pg failed to install with “Can’t find the libpq-fe.h header”. This was because PostgreSQL was not installed on the machine yet, which I fixed with brew install postgresql

Failed again as the rmagick gem failed to install. This was because ImageMagic was not installed on the system and was fixed with brew install imagemagick

Check bundler is working:

bundle -v or bundle exec rails s or bundle exec jekyll serve

Setting up Postgres

PostgreSQL was set up (see above) with brew install postgresql. This is what I did to get it set up.

To have launchd start postgresql at login:

ln -sfv /usr/local/opt/postgresql/*.plist ~Library/LaunchAgents

Then to load postgresql now:

launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist

Install Instrumentation:

psql postgres -c 'CREATE EXTENSION "adminpack";'

Create user:

psql postgres -c "CREATE USER username WITH PASSWORD 'password';"

Create Database:

psql postgres -c "CREATE DATABASE db_name WITH OWNER username;"

Grant privileges:

psql postgres -c "GRANT ALL PRIVILEGES ON DATABASE db_name TO username;"

Redis

When trying to launch the rails console/server I got a Sidkiq/Redis error as the machine did not have Redis installed. This was fixed with:

brew install redis
redis-server &

Tests

Setup Test Database

bundle exec rake db:migrate
bundle exec rake db:test:prepare
bundle exec rspec

Read more...

Useful Sublime Shortcuts and Commands

I use this post to remind me of keyboard shortcut keys, etc, for the Sublime Text 2 editor. These are just reminders for functionality that I am already somewhat familiar with, so there isn’t much explanation, which is readily available online.

Work in progress. Comment if you think there is somehing I should list here.

Searching

e.g -*/my_mobile_site/intermediate/*, -*/my_mobile_site/publish/*

Selection

Select Next

Ctrl D

Select Block/Column

By default this is set to ctrl+shift+up and ctrl+shift+down, but that did not work on my MacBook as those keys are already used by OS X. I therefore changed it, as shown below, to alt+shift+up and alt+shift+down.

To edit select_lines in Sublime:

  1. Go to Preferences > Key Bindings - User

  2. Add or edit the following lines, so that they look like this:

  { "keys": ["alt+shift+up"], "command": "select_lines", "args": {"forward": false} },
  { "keys": ["alt+shift+down"], "command": "select_lines", "args": {"forward": true} }

Check Key Bindings - Default to see what select_lines is set to by default.

Quic Open a File

Mac: CMD + P Ubuntu: Ctrl + P

Searches for a file in the current project based on the file name, press enter to open the selected file.

The Basics

Undo, Redo

Ctrl Z
Ctrl Shift Z

Read more...

Useful Postgres Commands

Login

psql -U username postgres

Show a list of Tables

\dt

Lists

Databases

\list

Users

\du

Quit

\q

Changing Owner and Privileges

The issues I came accross when trying to grant privileges/create a new database included that I had to be logged in as the owner of the databse and that I did not have permissions. The errors included:

  • `peer authentication failed for user “username”
  • `must be owner of database db_name
  • permission denied to drop role

You may be asked for a password, e.g. the deploy user’s password. If you log on using access keys, you may need to (re)set this:

sudo su
sudo passwd deploy

Log in as the user that owns the database, e.g. postgres (after exit to get back to the deploy account):

sudo -u postgres psql db_name postgres _db_name postgres

To change the database owner: ALTER DATABASE db_name OWNER TO username;

To grant privileges: GRANT ALL PRIVILEGES ON DATABSE database TO username;


Read more...

Useful Mac Commands

This post is a work in progress. Comment with suggestions of things to add.

Time

Manually Update Time

sudo ntpdate -vu time.apple.com

Read more...

Which Email Marketing Provider?

This is a feature list, feature comparison and brief overview for the main email marketing providers out there: Mailchimp (and Mandrill), Aweber and GetResponse.

Features important to us are:

  • Nurturing emails - automated campaigns AKA Auto Follow Up
  • Email open (and click on) stats AKA Analysis Reports and Google Analytics Integration
  • Update customer list/db from our website AKA API

Feature Comparison

  AWeber GetResponse Icontact Mailchimp Mandrill
Auto
Follow up
Y Y Y Y  
Analysis /
Reports
Y Y Y Y Y
Google
Analytics
Y Y Y Y Y
API Y Y Y Y Y
A/B
Testing
Y Y Y Y Y
Mobile
Friendly
Y Y Y Y  
Subscriber
Profiles
Y   Y Y  
Preview /
Testing
Y Y Y Y Y
Email
Editor
Y Y Y Y  
Subscriber
Segmenting
Y Y Y Y  
Signup
Forms
Y Y Y Y  
10k Monthly
Cost
$69 £45 $250* $75 **note

*Icontact advertise a plan for 10000 subscribers at £48.84 on their website, however a plan with all the features mentioned in the table would cost in the region of $250 per month.

**Mandrill charges per email, whereas the other providers charge per subscriber with unlimited emails to each subscriber. This is free for the first 12k emails, then $0.20 per 1000 emails for the next million emails, etc. So to send 100,000 emails in a month, you’d pay $17.60. See here for details.

Feature List

AWeber

AWeber has a well laid out feature list on their site (click on the “Learn More” links). They also have lots of info about their API, so I won’t duplicate either here.

GetResponse

GetResponse has a well laid out list of top features on their site. They also have some info about their API, so I won’t duplicate either here.

GetResponsse offer a landing page creator add-on wich allows you to create mobile ready landing pages, at an additional cost of £12/month.

When asked, this is what the GetResponse sales team said differentiates them from their competitors:

Besides the big difference in the pricing, as you can see yourself, we do have an array of features that we are constantly improving. Last year we released the new Landing Page creator, and soon we will be releasing the new Webform creator. We also have a 24/7 Customer Success team that is available for our customers.

Icontact

Icontact does not have a huge amount of information on their website, but were able to provide some details directly (as listed above and detailed below). For a starting point to using their “Open API”, see here.

Premier Account Features (the account which includes all the features listed in the comparison table) includes:

  • Business Discovery
  • Best Practices Review
  • Expert Message Assistance
  • Split A/B Testing
  • Strategy and Tactical Optimization
  • Reputation & Deliverability Management
  • Campaign Planning
  • Expert List Assistance
  • Training
  • Performance Feedback
  • Integration with Google Analytics
  • Dedicate Strategic Advisor For Email Marketing Assistance

Mailchimp

Mailchimp has a well laid out feature list on their site. They also have lots of info about their API, so I won’t duplicate either here.

Mandrill

Mandrill is an email infrastructure service, used to send automated one-to-one email like welcome messages, one-to-one marketing emails and customized newsletters.

Mandrill has a well laid out feature list and price calculator on their website. They also have lots of info about their API, so I won’t duplicate either here.


Read more...

Model Diagrams

Creating manual model diagrams (Entity-Relationship Diagrams) for rails applications is a laborious task, especially for more complex or bigger applications. There are fortunately some tools to generate these for you automatically.

These include rails-erd and railroady.

Rails-ERD is easy to use. Simply install Graphviz, add the gem and run the rake erd take task, as explained here.

This post is not finished, more details will be added (about railroady and/or similar tools), and this is expected to be done by 2025 at the latest.


Read more...