Skip to content
This repository was archived by the owner on Feb 7, 2018. It is now read-only.

How to analyze your pipe runs with Bunyan

Jump to bottom
David Taieb edited this page Aug 4, 2015 · 7 revisions

Introduction
In this post, we'll discuss how the Simple Data Pipe leverages the Bunyan nodejs logging framework (https://github.com/trentm/node-bunyan) to capture detailled logging information about a pipe run. Reports are then automatically attached to a Cloudant database for further analysis. We'll then show how to analyze the report using the Bunyan viewer tool

A word about Bunyan
Bunyan is a simple and fast JSON logging library for node.js services. It can be configured to output the data to streams that can be stored anywhere. Simple Data pipes is using this library to capture log information about a particular run, then attach the report to the pipe run document stored in the pipe_db Cloudant database. This logging framework support many log levels: trace, debug, info, warning, error.
Bunyan also provide a CLI utility to view the log in a pretty formatted manner with the ability to filter by logging group and level. See https://github.com/trentm/node-bunyan for more information.

**How to locate the log for a particular run **
The scenario is as follow: You attempted to perform a pipe run and something went wrong. You now need to locate the log, download it from the pipe_db Cloudant database and analyze it for troubleshooting.

  1. Locate the log: Go to bluemix and click on your pipe app instance
  2. Click on the pipe-cloudant-service box to open the Cloudant dashboard, then click on the Launch button
  3. In the Cloudant dashboard, click on the pipe_db database
  4. On the left tab, click on _design/application, then Views, then all_runs
  5. On the left hindsight of the all_runs view, you can see all the runs documents sorted in increasing order by date. Therefore the run you are looking for is last in the view (you may need to page a few times if you have a lot of runs in your database).
  6. Click on the pencil icon to open the run document. You should be able to see the json object for the run.
  7. Click on the View Attachments dropdown button and right-click on run.log menu. Then click on the Save Link As... menu. This will download the file in a directory of your choice.
  8. The next step will be to use the Bunyan CLI tool to analyze run.log.

Clone this wiki locally