new cookbook: fb_networkmanager (facebook#127)

Summary:
This cookbook aims to manage NetworkManager in the most controlled
way possible. It might-could-should be sent over to CPE cookbooks,
I can redirect over there if preferred.

While, much to my dismay, some people do run NM on servers, and this
will work for that, it was built to handle this client machine case.

NetworkManager treats its config files as databases and can change them,
so they can't be handled simply as templates - more finesse is needed.

However, write-once doesn't provide the level of management that
FB-API-style cookbooks expect. This marries the two. All desired
settings are managed 100%, and NetworkManager is allowed to make
additive settings (such as "seen-bssids" or "password" [sic]) so that it
can continue to function for users.

It is immune to the format changes that NM can make - spacing and
ordering changes will not cause the file to be updated any more than
ignorable settings. When updating the file, care is taken to stick
close to NM's style.

I considered making blacklist/whitelists of fields that NM could
modify, but in practice since we merge in the user's config, the user's
config is effectively a blacklist, and whitelist would likely be far
too fragile.

It has full docs, including on it's Migration Feature which will be
useful for anyone who had their own Chef to do this but wants to
migrate.

It also comes with pretty good test coverage.

Pull Request resolved: facebook#127

Differential Revision: D69685647

fbshipit-source-id: 45632b1015565a87cc515f2b74d3dd15a6cf9894

Author: jaymzh

cookbooks/fb_init_sample/metadata.rb

@@ -47,6 +47,7 @@
 depends 'fb_mlocate'
 depends 'fb_modprobe'
 depends 'fb_motd'
+depends 'fb_networkmanager'
 depends 'fb_nscd'
 depends 'fb_nsswitch'
 depends 'fb_postfix'

cookbooks/fb_init_sample/recipes/default.rb

@@ -82,6 +82,9 @@
 # until we defined a UID_MAP that works with testing, this can't
 # run in kitchen tests
 # include_recipe 'fb_users'
+if node.debian? || node.ubuntu?
+  include_recipe 'fb_networkmanager'
+end
 if node.centos?
   # We turn this off because the override causes intermittent failures in
   # Travis when rsyslog is restarted

cookbooks/fb_networkmanager/README.md

@@ -0,0 +1,238 @@
+fb_networkmanager Cookbook
+============================
+An attribute-driven API to configure Network Manager
+
+Requirements
+------------
+
+Attributes
+----------
+* node['fb_networkmanager']['config']
+* node['fb_networkmanager']['enable']
+* node['fb_networkmanager']['system_connections']
+* node['fb_networkmanager']['system_connections'][$NAME]['_defaults']
+* node['fb_networkmanager']['system_connections'][$NAME]['_migrate_from']
+* node['fb_networkmanater']['extra_packages']
+* node['fb_networkmanater']['manage_packages']
+
+Usage
+-----
+### Packages
+
+By default this cookbook will install the network-manager package for your
+platform. If you intend to make connections that require optional packages,
+such as VPN connections, you can add the necessary packages to the array in
+`node['fb_networkmanater']['extra_packages']` so that they will be installed
+early enough.
+
+If you'd prefer to manage packages yourself you may set
+`node['fb_networkmanater']['manage_packages']` to `false`.
+
+### Config
+
+The global config (`/etc/NetworkManager/NetworkManager.conf`), is controlled
+by the `config` hash. It's a two-level hash where the top-level is INI section
+names and the second level is key-value pairs for the options in that section.
+
+For example:
+
+```ruby
+node.default['fb_networkmanager']['config']['main']['foo'] = 'bar'
+```
+
+would render as:
+
+```text
+[main]
+foo=bar
+```
+
+The default config is based on the Ubuntu config but should be safe for all
+distros.
+
+### System Connections
+
+Network Manager unfortunately uses the files in the `system-connections` folder
+as a data store about those networks. This means they can change out from under
+you as it addds its own information. For example, for WiFi entries, it can add
+BSSIDs it has seen to the file.
+
+As such using the desired config to populate a template is not sufficient - this
+would both lose data that Network Manager wants and also cause a lot of
+unnecessary resource firing.
+
+To work around this scenario, this cookbook loads in the existing file, merges
+in the desired config, and then checks to see if the resulting contents are
+different from just the loaded file. If the values would not change, then we do
+not write out the new config. If the values are different, then we write out
+the merged config. Since Network Manager can write out values in a different
+order or with different spacing, we dont' compare the actual files, but instead
+the parsed data. We leverage IniParse for reading and writing INI files since
+it is bundled with Chef.
+
+All that said, the system_connections hash works a lot like the `config` hash,
+except there's an extra level for each connection. For example:
+
+```ruby
+node.default['fb_networkmanager']['system-connections']['mywifi'] = {
+  'connection' => {
+    'type' => 'wifi',
+    'id' => 'Cool Wifi',
+    'uuid' => '...',
+  },
+  'wifi' => {
+    'mode' => 'infrastructure',
+    'ssid' => 'Cool Wifi',
+  },
+  'wifi-security' => {
+    'auth-alg' => 'open',
+    'key-mgmt' => 'wpa-psk',
+    'psk' => 'SuperS3kr1t',
+  },
+}
+```
+
+Would create `/etc/NetworkManager/system-connections/fb_networkmanager_mywifi`
+with this content:
+
+```text
+[connection]
+id=Cool Wifi
+uuid=...
+type=wifi
+
+[wifi]
+mode=infrastructure
+ssid=Cool Wifi
+
+[wifi-security]
+auth-alg=open
+key-mgmt=wpa-psk
+psk=SuperS3kr1t
+```
+
+Note that all files we make are prefixed with `fb_networkmanager`, so that we
+can cleanup files we created that are no longer in the config.
+
+### A note on booleans
+
+It is worth noting that various plugins and parts of the config expect
+different kinds of booleans - some `true` and `false`, others `yes` and `no`.
+Normally, an FB Attribute API cookbook would take a true ruby boolean and
+convert it to the appropriate string for a system, but since it's not
+consistent across NM, we leave it to the user to specify the right one for the
+right value. This is true both in `config` and in `system_connections`.
+
+### A note on UUIDs
+
+We generally recommend coming up with a static UUID per connection you want
+to rollout. For example, generate a UUID (for example using `uuidgen`, or
+by `cat /proc/sys/kernel/random/uuid`), and then associate that with given
+connection, statically in your config. You must use a different UUID for each
+connection (obviously), but using the same UUID for the same connection across
+machines makes debugging easier.
+
+However, if you want truly unique UUIDs, one option is to just not specify a
+UUID and let Network Manager fill one in. However, not all versions of NM
+support this, and some will just ignore that connections.
+
+You can't just generate UUIDs in the recipe, as they'll change on every run. So
+here's one way to solve that problem: build each UUID seeded with the hostname
+and the connection name so they stay the same across runs:
+
+```ruby
+node.default['fb_networkmanager']['system-connections']['mywifi'] = {
+  'connection' => {
+    'type' => 'wifi',
+    'id' => 'Cool Wifi',
+    'uuid' => UUIDTools::UUID.sha1_create(
+      UUIDTools::UUID_DNS_NAMESPACE,
+      "#{node['fqdn']}/Cool Wifi",
+    ),
+  },
+  ...
+}
+```
+
+#### Migrating from existing configs
+
+Migrating to this cookbook could potentially pose a problem: you want all the
+information from the existing connection, but you don't want a duplicate
+connection.
+
+We provide a `_migrate_from` key. When populated, we'll use that as our base
+config the first time, merge any data provided in the node, and then delete
+the old config.
+
+This provides seemless transition - it will preserve the UUID, which will
+keep network manager from thinking any connections went away, and ensure
+in-use connections don't drop.
+
+For example, let's say you had droped a file
+`/etc/NetworkManager/system-connections/OurCorpWifi` that you had dropped off
+with `cookbook_file`, or a script, or even that you had was built through
+manually setting it up in the NM GUI. You could then do:
+
+```ruby
+node.default['fb_networkmanager']['system-connections']['our_corp_wifi'] = {
+  '_migrate_from' => 'OurCorpWifi',
+  'connection' => {
+    'type' => 'wifi',
+    'id' => 'OurCorpWifi',
+  },
+  'wifi' => {
+    'mode' => 'infrastructure',
+    'ssid' => 'OurCorpWifi',
+  },
+  'wifi-security' => {
+    'auth-alg' => 'open',
+    'key-mgmt' => 'wpa-psk',
+    'psk' => 'SuperS3kr1t',
+  },
+}
+```
+
+Then anything not specified here will be pulled in from the existing
+`OurCorpWifi` file. Note that any settings that you care about should be
+specified in the node to ensure that on new setups, you're not missing critical
+configuration.
+
+Note that if the original service file isn't there, Chef will just create a new
+connection file (though it will warn). Also note that once Chef has created one,
+it stop pulling in the old file (and will remove it).
+
+#### Providing defaults
+
+In general, the Chef config wins over the user config as described above.
+However it is often desirable to specify a default config in case the user does
+not specify anything. In that case you can use the `_defaults`. For example:
+
+```ruby
+node.default['fb_networkmanager']['system-connections']['our_corp_wifi'] = {
+  '_migrate_from' => 'OurCorpWifi',
+  '_defaults' => {
+    'connection' => {
+      'autoconnect-priority' => '100',
+    }
+  },
+  'connection' => {
+    'type' => 'wifi',
+    'id' => 'OurCorpWifi',
+  },
+  'wifi' => {
+    'mode' => 'infrastructure',
+    'ssid' => 'OurCorpWifi',
+  },
+  'wifi-security' => {
+    'auth-alg' => 'open',
+    'key-mgmt' => 'wpa-psk',
+    'psk' => 'SuperS3kr1t',
+  },
+}
+```
+
+The rendered config here will set `connection.autoconnect-priority` to 100
+if there is no value for it found in the existing file, but will use the value
+in the file if it exists. The logic here is quite simple:
+
+  defaults < user config < chef config

cookbooks/fb_networkmanager/attributes/default.rb

@@ -0,0 +1,41 @@
+#
+# Cookbook:: fb_networkmanager
+# Recipe:: default
+#
+# Copyright (c) 2020-present, Vicarious, Inc.
+# Copyright (c) 2020-present, Facebook, Inc.
+# All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+default['fb_networkmanager'] = {
+  'enable' => false,
+  'manage_packages' => true,
+  'extra_packages' => [],
+  'system_connections' => {},
+  'config' => {
+    'main' => {
+      'plugins' => [
+        'ifupdown',
+        'keyfile',
+      ],
+    },
+    'ifupdown' => {
+      'managed' => false,
+    },
+    'device' => {
+      'wifi.scan-rand-mac-address' => false,
+    },
+  },
+}

cookbooks/fb_networkmanager/libraries/default.rb

@@ -0,0 +1,61 @@
+#
+# Cookbook:: fb_networkmanager
+# Recipe:: default
+#
+# Copyright (c) 2020-present, Vicarious, Inc.
+# Copyright (c) 2020-present, Facebook, Inc.
+# All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+require 'iniparse'
+
+module FB
+  class Networkmanager
+    def self.active_connections
+      return {} unless ::File.exist?('/usr/bin/nmcli')
+
+      s = Mixlib::ShellOut.new('nmcli -t conn show --active').run_command
+      return {} if s.error?
+
+      cons = {}
+      s.stdout.each_line do |line|
+        name, uuid, type, device = line.strip.split(':')
+        cons[name] = {
+          'uuid' => uuid,
+          'type' => type,
+          'device' => device,
+        }
+      end
+      cons
+    end
+
+    def self.to_ini(content, array_sep)
+      IniParse.gen do |doc|
+        content.each_pair do |sect, opts|
+          doc.section(sect, :option_sep => '=') do |section|
+            opts.each_pair do |opt, val|
+              v = [val].flatten.join(array_sep)
+              # for multi-value
+              if val.is_a?(Array)
+                v << array_sep
+              end
+              section.option(opt, v)
+            end
+          end
+        end
+      end.to_s
+    end
+  end
+end