|
1 | | -# fixed-length-file-handler |
2 | | -Handlers for Fixed Length/Fixed Width files in a beautiful Kotlin DSL |
| 1 | +# Fixed Length File Handler |
| 2 | + |
| 3 | + |
| 4 | + |
| 5 | +## Introduction |
| 6 | +When processing data from some systems (mainly legacy ones), it's usual to have Fixed Length Files, which are files that contain lines which content is split using a specific length for each field of a record. |
| 7 | + |
| 8 | +This kind of files are sometimes tricky to handle as many times there is a spaghetti of string manipulations and padding, and character counting and... Well, many things to take care of. |
| 9 | + |
| 10 | +This library comes to the rescue of programmers dealing with fixed length files. It enables you to simply define how your records are structured and it will handle these records for you in a nice Kotlin DSL for further processing. |
| 11 | + |
| 12 | +## Using with Gradle |
| 13 | + |
| 14 | +This library is published to `Bintray jcenter`, so you'll need to configure that in your repositories: |
| 15 | +```kotlin |
| 16 | +repositories { |
| 17 | + mavenCentral() |
| 18 | + jcenter() |
| 19 | +} |
| 20 | +``` |
| 21 | + |
| 22 | +And then you can import it into your dependencies: |
| 23 | +```kotlin |
| 24 | +dependencies { |
| 25 | + implementation("br.com.guiabolso:fixed-length-file-handler:{version}") |
| 26 | +} |
| 27 | +``` |
| 28 | + |
| 29 | +## Basic Usage |
| 30 | + |
| 31 | +The basic usage assumes that you're reading a file with a single type of record. |
| 32 | + |
| 33 | +Given a Fixed-Length File: |
| 34 | + |
| 35 | + |
| 36 | +#### Definition |
| 37 | + |
| 38 | +| Field | Type | Initial Position | Final Position Exclusive | |
| 39 | +| ----- | ---- | ---------------- | ------------------------ | |
| 40 | +| UserName | String | 0 | 30 | |
| 41 | +| User Document | Int | 30 | 39 | |
| 42 | +| User Registry Date | LocalDate | 39 | 49 | |
| 43 | + |
| 44 | +#### File |
| 45 | + |
| 46 | +``` |
| 47 | +FirstUsername 1234567892019-02-09 |
| 48 | +SecondAndLongerUsername 9876543212018-03-10 |
| 49 | +ThirdUsernameWithShorterDoc 0000001232017-04-11 |
| 50 | +``` |
| 51 | + |
| 52 | +We can parse it with the `fixedLengthFileParser` DSL: |
| 53 | + |
| 54 | +```kotlin |
| 55 | +data class MyUserRecord(val username: String, val userDoc: Int, val registryDate: LocalDate) |
| 56 | + |
| 57 | +val fileInputStream: InputStream = getFileInputStream() |
| 58 | + |
| 59 | +fixedLengthFileParser<MyUserRecord>(fileInputStream) { |
| 60 | + MyUserRecord( |
| 61 | + field(0, 30, Padding.PaddingRight(' ')), |
| 62 | + field(30, 39, Padding.PaddingLeft('0')), |
| 63 | + field(39, 49) |
| 64 | + ) |
| 65 | +} |
| 66 | +``` |
| 67 | + |
| 68 | +The library is prepared to handle `Left Padding` and `Right Padding`. It's also prepared to handle many of Kotlin/Java types. |
| 69 | + |
| 70 | +## Closing the file stream |
| 71 | + |
| 72 | +**Attention** - You're responsible for closing the stream after processing the sequence, so be sure to close it! |
| 73 | + |
| 74 | +## Default parsing |
| 75 | + |
| 76 | +This library is prepared to handle some of the most usual Kotlin/Java types. More types may be added if they're required. The default types are: |
| 77 | + |
| 78 | +- String |
| 79 | +- Int |
| 80 | +- Double |
| 81 | +- Long |
| 82 | +- Char |
| 83 | +- Boolean (Case insensitive) |
| 84 | +- LocalDate (Using default DateTimeFormatter) |
| 85 | +- LocalDateTime (Using default DateTimeFormatter) |
| 86 | +- BigDecimal |
| 87 | + |
| 88 | +## Custom parsing |
| 89 | + |
| 90 | +There might be times where the default types are not enough, and you need a custom parser for a given record. |
| 91 | + |
| 92 | +For example: You know that a specific number contains a currency, and the last two digits are used for the cents. |
| 93 | + |
| 94 | +This library is prepared to handle cases where you need custom parsing for a String, by modifying the `field` invocation: |
| 95 | + |
| 96 | +```kotlin |
| 97 | + |
| 98 | +// Parsing the field 0000520099 to 5200.99 |
| 99 | + |
| 100 | +field(15, 25, Padding.PaddingLeft('0')) { str: String -> StringBuilder(str).insert(str.length - 2, ".").toString().toBigDecimal() } |
| 101 | +``` |
| 102 | + |
| 103 | +## Advanced Usage |
| 104 | + |
| 105 | +For an unknown reason, many Fixed-Length file providers use the same file for more than one record, denoting a specific bit for record identification, so there's a possibility that this happens: |
| 106 | + |
| 107 | +``` |
| 108 | +1 FirstUserName 123.12 |
| 109 | +1 SecondUserName 002.50 |
| 110 | +2 123456789 2019-02-09UserDocs |
| 111 | +2 000812347 2018-03-08AnotherUserDocs |
| 112 | +``` |
| 113 | + |
| 114 | +In this cases, the software must look at the first `char` to determine the record type. This situation is usually what leads to a spaghetti string manipulation. We can solve it by using this library's "advanced" options: |
| 115 | + |
| 116 | +```kotlin |
| 117 | +data class FirstRecordType(username: String, userMoney: BigDecimal) |
| 118 | +data class SecondRecordType(userCode: Int, registerDate: LocalDate, docs: String) |
| 119 | + |
| 120 | +fixedLengthFileParser<Any>(fileInputStream) { |
| 121 | + withRecord({ line -> line[0] == '1' }) { |
| 122 | + FirstRecordType( |
| 123 | + field(2, 22, Padding.PaddingRight(' ')), |
| 124 | + field(22, 28, Padding.PaddingLeft('0')) |
| 125 | + ) |
| 126 | + } |
| 127 | + |
| 128 | + withRecord( { line -> line[0] == '2' }) { |
| 129 | + SecondRecordType( |
| 130 | + field(2, 15, Padding.PaddingRight(' ')), |
| 131 | + field(15, 25), |
| 132 | + field(25, 40, Padding.PaddingRight(' ')) |
| 133 | + ) |
| 134 | + } |
| 135 | +} |
| 136 | +``` |
| 137 | + |
| 138 | +## Features |
| 139 | + |
| 140 | +- The file is streamed into a sequence of values, and is never loaded in its entirety to the memory. You should expect this to have a good performance over a very big file. |
| 141 | +- The Kotlin DSL makes it easier to define the file parsing in a single point, and the sequence processing can be done anywhere |
0 commit comments