Merge branch 'release/2.3.0'

Author: mblaschke

CHANGELOG.md

@@ -1,6 +1,25 @@
 CliTools Changelog
 ==================
 
+next - UPCOMING
+------------------
+
+
+2.3.0 - 2016-01-26
+------------------
+- Updated compile documentation
+- Improved compile scripts
+- Added rsync.opts documentation
+- Added rsync.opts section in `clisync.yml` for additional rsync options
+- No terminal title if stdout redirect is detected
+- Added `docker:cleanup` for cleanup of orphaned images and volumes
+- Renamed `sync:server` to `sync`
+- Renamed `sync:backup` to `share:backup`
+- Renamed `sync:restore` to `share:restore`
+- Renamed `sync:deploy` to `deploy`
+- Added `COMPOSER=custom.json ct php:composer` support
+- Support for new PHP and TYPO3 Docker Boilerplate (with APPLICATION_USER detection, configurable)
+
 2.2.0 - 2015-08-19
 ------------------
 - Introduced `docker:up --switch` (`docker:up` without --switch parameter no longer shutdown previous one)

Documentation/ALIASES.md

@@ -22,6 +22,7 @@ alias dcroot='ct docker:root'
 
 # Execute predefined cli in docker container
 alias dccrun='ct docker:cli'
+alias dcrun='ct docker:cli'
 
 # Run command
 alias dcexec='ct docker:exec'

Documentation/COMMANDS.md

@@ -77,7 +77,7 @@ All log commands are using a grep-filter (specified as optional argument)
 |                            | __ct mysql:restore typo3 dump.sql.xz__ -> xz'ed (lzma'ed) sql dump        |
 |                            | __ct mysql:restore typo3 dump.sql --filter=typo3__ -> No TYPO3 cache tables in dump |
 | ct mysql:restore           | Create (and drops if already exists) a database and restore from a dump   |
-|                            | Dump file can be plaintext, gziped, bzip2 or lzma compressed              |
+|                            | Dump file can be plaintext, gzip, bzip2 or lzma compressed              |
 |                            | and will automatically detected                                           |
 |                            | __ct mysql:restore typo3 dump.sql.bz2__                                   |
 | ct mysql:convert           | Convert character set and collation of a database                         |

Documentation/Examples/clisync.yml

@@ -53,6 +53,9 @@ GLOBAL:
         # set target as sub directroy (will be appended to working directory)
         workdir: ""
 
+        # Additional options ("-rlptD --delete-after --progress --human-readable" is already set)
+        opts: ""
+
         # exclude list/patterns for files and directories
         exclude:
             # Temp files
@@ -61,9 +64,7 @@ GLOBAL:
 
             # VCS
             - ".git*"
-            - ".gitignore"
-            - ".gitmodules"
-            - ".svn"
+            - ".svn*"
 
             # Build files
             - "composer.json"

Documentation/INSTALL.md

@@ -1,4 +1,4 @@
-q[<-- Back to main section](../README.md)
+[<-- Back to main section](../README.md)
 
 # Installation
 
@@ -17,8 +17,13 @@ q[<-- Back to main section](../README.md)
   - sudo
   - moreutils (ifdata)
   - coreutils (grep, sort, uniq, awk, cat, df, ip, cut, lsb_release, wall)
+  - rsync (to prevent bugs with umlauts while you have to use rsync with version >= 3.0 (GNU version))
+  - gnu-sed
   - docker and docker-compose (if you want to use docker)
   - mysql (if you want to use mysql)
+  
+When you're want to using clitools under OSX, you could use homebrew, an excellent package manager
+to easily install the above mentioned requirements.
 
 
 ## Install clitools
@@ -36,37 +41,9 @@ wget -O"$HOME/.clitools.ini" https://github.com/webdevops/clitools/blob/develop/
 
 ## Aliases
 
-Now you can use following aliases (some aliases requires clitools 1.8.0!):
+Now you can use some useful aliases (some aliases requires clitools 1.8.0!):
 
-```bash
-# Shortcut for auto-tree-searching make
-alias make='ct make'
-
-# Shortcut for auto-tree-searching make
-alias composer='ct php:composer'
-
-# Shortcut for docker-compose (autosearch docker-compose.yml in up-dir, you don't have to be in directory with docker-compose.yml)
-alias dcc='ct docker:compose'
-
-# Startup docker-container (and shutdown previous one, v1.9.0 and up)
-alias dccup='ct docker:up'
-alias dccstop='ct docker:compose stop'
-
-# Enter main docker container (as CLI_USER if available - if not specified then root is used)
-alias dcshell='ct docker:shell'
-alias dcsh='ct docker:shell'
-
-# Enter main docker container (as root)
-alias dcroot='ct docker:root'
-
-# Execute predefined cli in docker container
-alias dccrun='ct docker:cli'
-alias dcrun='ct docker:cli'
-
-# Execute mysql client in docker container
-alias dcsql='ct docker:mysql'
-alias dcmysql='ct docker:mysql'
-```
+[Example aliases for clitools](ALIASES.md)
 
 ## Configuration
 
@@ -135,9 +112,12 @@ cd clitools
 make all
 ```
 
+### Details about the Makefile
 If you take a look into the `Makefile`, you will see which tasks have been executed.
 The steps are:
 
 1. Execute composer command
 2. Start main build script
 3. Copy the previously built phar file to `/usr/local/bin`, so that you can execute clitools just by typing ct.
+
+*Note*: If you're compiling clitools in OSX the easiest thing is to install requirements with homebrew: `brew install coreutils g-sed homebrew/php/box`

Documentation/USAGE-DOCKER.md

@@ -4,7 +4,7 @@
 
 ## Docker creation
 
-You can easly create new docker instances (from my or a custom docker boilerplate) also with code intalization
+You can easily create new docker instances (from my or a custom docker boilerplate) also with code initialization
 and Makefile running.
 
 ```bash
@@ -14,6 +14,9 @@ ct docker:create foobar
 # Create and start new docker boilerplate into foobar directory
 ct docker:create foobar --up
 
+# Create new typo3 docker boilerplate
+ct docker:create foobar --docker=typo3
+
 # Create new custom docker boilerplate 
 ct docker:create foobar --docker=git...
 
@@ -29,8 +32,8 @@ ct docker:create foobar --code=git... --make=build --up
 
 ## Docker startup
 
-The `docker:up` command will search the `docker-compose.yml` in the current parent directroy tree and
-execute `docker-compose` from this directroy - you don't have to change the current directroy.
+The `docker:up` command will search the `docker-compose.yml` in the current parent directory tree and
+execute `docker-compose` from this directory - you don't have to change the current directory.
 
 Also the previous docker instance will be shut down to avoid port conflicts.
 
@@ -42,7 +45,7 @@ ct docker:up
 ## Custom docker commands
 
 As `docker:up` the `docker:compose` will search the `docker-compose.yml` and will execute your command
-from this directroy.
+from this directory.
 
 ```bash
 # Stop docker instance
@@ -83,7 +86,7 @@ ct docker:exec ps
 
 You can define a common CLI script entrypoint with the environment variable CLI_SCRIPT in your docker containers.
 The environment variable will be read by `ct docker:cli` and will be executed - you don't have to jump
-into your containers, you can start your CLI_SCRIPTs from the outide.
+into your containers, you can start your CLI_SCRIPTs from the outside.
 
 ```bash
 # Execute predefined cli command with argument "help" in "main" container
@@ -105,3 +108,21 @@ ct docker:sniff http --full
 # Show mysql querys by using network sniffer
 ct docker:sniff mysql 
 ```
+
+## Docker cleanup
+Docker currently doesn't cleanup orphaned images or volumes so you have to cleanup your system regularly to get free disk space
+
+```bash
+ct docker:cleanup
+```
+
+It's a shortcut for:
+
+```bash
+# Docker image cleanup:
+docker images | grep "<none>" | awk "{print \$3}" | xargs --no-run-if-empty docker rmi -f
+
+# Docker volume cleanup
+docker pull martin/docker-cleanup-volumes
+docker run -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker --rm martin/docker-cleanup-volumes
+```

Documentation/USAGE-MYSQL.md

@@ -17,7 +17,7 @@ ct mysql:list
 
 # Debugging
 
-The `ct mysql:querylog` and `ct mysql:slowlog` provides a convinent way to access the general query log 
+The `ct mysql:querylog` and `ct mysql:slowlog` provides a convenient way to access the general query log 
 and the slow log.
 
 In the query log you can see all queries send and executed by the MySQL database.
@@ -67,7 +67,7 @@ Also the compression is automatically detected by file mime type.
 ct mysql:restore typo3 dump.sql
 ```
 
-# Datbase charset conversion
+# Database charset conversion
 
 ```bash
 # Convert database to UTF8

Documentation/USAGE-PHP.md

@@ -4,7 +4,7 @@
 
 ## Composer (auto searching in path tree)
 
-Because you always need to jump into `composer.json` directroy `ct php:composer` will do this for you
+Because you always need to jump into `composer.json` directory `ct php:composer` will do this for you
 
 ```bash
 # Run composer install task

Documentation/USAGE-SYNC.md

@@ -11,7 +11,7 @@ First you need to create a `clisync.yml` in your project directory, the CliTools
 of this file by using following command:
 
 ```bash
-ct sync:init
+ct sync-init
 ```
 
 If you need special SSH settings (ports, compression, identify...) please use your `~/.ssh/config` file 
@@ -30,59 +30,78 @@ Now you can sync your `production` server to your local installation:
 
 ```bash
 # Full sync (files and database, with interactive context selection)
-ct sync:server
+ct sync
 
 # Only MySQL (from production context, without interactive context selection)
-ct sync:server production --mysql
+ct sync production --mysql
 
 # Only Files (from production context, without interactive context selection)
-ct sync:server production --rsync
+ct sync production --rsync
 ```
 
-## Project sharing  (ct sync:backup and ct sync:restore)
+## Project sharing  (ct share:backup and ct share:restore)
 
 The sharing can be used to share files (assets) and databases between developers.
 Please use a common development/storage server with ssh access for each developer for this feature.
 
 ```bash
 # Make backup of current state and transfer to share server
-ct sync:backup
+ct share:backup
 
 # ... only MySQL
-ct sync:backup --mysql
+ct share:backup --mysql
 
 # ... only files
-ct sync:backup --rsync
+ct share:backup --rsync
 
 # Restore to state from the share server
-ct sync:restore
+ct share:restore
 
 # ... only MySQL
-ct sync:restore --mysql
+ct share:restore --mysql
 
 # ... only files
-ct sync:restore --rsync
+ct share:restore --rsync
 
 ```
 
 ## Lightweight deployment (ct sync:deploy)
 
 With `sync:deploy` you can push your files to your production servers.
 Please keep in mind that this feature is just an wrapped rsync and should only be
-the simplest solution for deployment. For more advanced or centralized deployemnt try
+the simplest solution for deployment. For more advanced or centralized deployment try
 solutions build on Jenkis, Ansible and others.
 
 ```bash
 # Push your project to your servers (with interactive context selection)
-ct sync:deploy
+ct deploy
 
 # Push your project to your staging server (without interactive context selection)
-ct sync:deploy staging
+ct deploy staging
 ```
 
+## Rsync additional options (rsync.opts)
+
+The rsync options `-rlptD --delete-after --progress --human-readable` are automatically added when using the sync
+features of clitools. In addition, it is possible to set more options by adding them via `rsync.opts` (available since
+version 2.3 of clitools). In the following example the option `--iconv=UTF8-MAC,UTF8` is added to enable rsync filename
+charset conversion between local and remote file system. You need this especially when using rsync between hosts where a
+charset conversion on filenames is necessary (see `man rsync` for further details; e.g.: you probably need this when
+you sync between mac and linux utf-8 filesystem).
+
+```
+...
+# Enable filename charset conversion where local is UTF8-MAC (Mac OSX) and remote is UTF8 (e.g. Linux)
+rsync.opts: "--iconv=UTF8-MAC,UTF8"
+...
+```
+
+If you run into problems with `--iconv` make shure you have a compatible version of rsync >= 3.0 (as already mentioned
+in the [install section](INSTALL.md) of this documentation).
+
 ## Advanced ssh options
 
-If you need some advaned ssh options (eg. other ports) use your `~/.ssh/config` configuration file:
+If you need some advanced ssh options (eg. other ports) use your `~/.ssh/config` configuration file:
 
     Host project-server
         Hostname project-server.example.com

Makefile

@@ -9,5 +9,8 @@ install:
 autoload:
 	sh -c "cd src ; composer dump-autoload --optimize --no-dev"
 
+update:
+	sh -c "cd src ; composer update"
+
 sloccount:
 	sloccount src/app/ src/command.php