etch-users — General discussion and questions about using etch

[etch-users] Moving etch 4.0.0 from Ubuntu 18.04 to Ubuntu 20.04 client

[John H. <hil...@gm...>]

I haven't had much involvement with etch because it's always worked. I've
never used this list for support, and I'm not sure if this list is still
used based on the date of the last activity. Figured I would reach out in
case somebody can help point me in the right direction.

I recall zipping/unzipping the etch 4.0.0 directory when we updated from
Ubuntu 14.04 to 18.04 without issue. I'm doing the same thing for 20.04. I
installed Ruby, and see that Ruby changed from 2.5.1 to 2.7 something.

I get the following error when etch attempts to check in:
Neither PUB key nor PRIV key: nested asn1 error
Traceback (most recent call last):
4: from /usr/local/etch-4.0.0/client/bin/etch:115:in `<main>'
3: from /usr/local/etch-4.0.0/client/lib/etch/client.rb:541:in
`process_until_done'
2: from /usr/local/etch-4.0.0/client/lib/etch/client.rb:2586:in `sign_post!'
1: from /usr/local/etch-4.0.0/client/lib/etch/client.rb:2586:in `new'
/usr/local/etch-4.0.0/client/lib/etch/client.rb:2586:in `initialize':
Neither PUB key nor PRIV key: nested asn1 error (OpenSSL::PKey::RSAError)

I may try to install the older version of ruby once I create a backup of my
dev work thus far. I have no idea if ruby even has anything to do with this
error. . Thanks.

[etch-users] Etch 5.0.0 released

[Jason H. <jh...@ap...>]

The big change from the last release (4.1.0) is that etch now supports YAML in place of all of the previous XML files.  So individual files can be configured with config.yml instead of config.xml.  nodes.yml is an alternative to nodes.xml, defaults.yml instead of defaults.xml, etc.  For now all of the XML files are also still supported.  If y’all generally switch your installs to YAML we can consider dropping XML in a future release (6.0 perhaps?)

All of the wiki documentation has been updated:  https://github.com/etch/etch/wiki

Etch has long had a reasonably good integration test suite that fires up an etch server and uses the etch client to test things.  I’ve started to add unit testing of the server which allows finer grained testing.  It has allowed me to find and fix a number of bugs.  Coverage if not complete yet but every little bit helps.

Looks like I also need to back up, since I created a 4.1.0 release but didn’t announce it.  Looks like the big change there from 4.0.1 was moving from Rails 3 to Rails 4.  I also changed the column in the database that is used to store the configuration for a given file on a particular client to a compressed long blob so that it can hold configuration that is larger than 64k.

Thanks a bunch to Pat O’Brien for his contributions and for pushing me to get this released.

[etch-users] Etch release: 4.0.1

[Jason H. <jh...@ap...>]

Well, it has been almost 2 years since the last etch release.  Must be time…  :)

Since the last release all development has moved to Github.  Code, tickets and the wiki documentation are now on Github at https://github.com/etch/etch

The only thing left on Sourceforge are the mailing lists.  I don’t see any particularly compelling reason to move them given the incredibly low traffic.  But we could move them to Google groups or the like at some point.

Releases can be found at https://github.com/etch/etch/releases

Looking through the commits since 4.0.0 I think these are the major changes:

* Bump rails to 3.2.16
* Re-initialize global vars after excution of external sources (Jae Park)
* XML and JSON output of client summary (Jae Park)
* Fix encoding under ruby 1.9


Here’s the full commit log since the last release:

commit 124fa33cc3393b213df25bab5508b0c5d74e2cfb
Author: Jason Heiss <jh...@ap...>
Date:   Tue Feb 11 17:49:32 2014 -0500

    Bump rails to 3.2.16

commit 378ec50b22891d509770fce825d7c61d60a0c2d9
Author: Jason Heiss <jh...@ap...>
Date:   Tue Feb 11 17:49:20 2014 -0500

    Add comment

commit 1045e2181f32f879abbed8fe47b26f832d20091c
Author: Jason Heiss <jh...@ap...>
Date:   Tue Feb 11 16:26:15 2014 -0500

    Update README

commit 626fcfc14b718c763748c7c5e969b99f43af134f
Merge: 92f295c 5daa6c3
Author: Jason Heiss <jh...@ap...>
Date:   Fri Oct 11 05:40:18 2013 -0700

    Merge pull request #19 from jaeheung90/load_path
    
    Re-initialize global vars after excution of external sources

commit 5daa6c326eb64743a7f8f42b7f7a61b4c4ca985a
Author: Jae Park <jae...@gm...>
Date:   Thu Oct 10 18:46:37 2013 -0700

    Re-initialize global vars after excution of external sources

commit 92f295cd7a5fd0261c7c05333c324b5d10ced223
Merge: 34dc6c6 02aaa34
Author: Jason Heiss <jh...@ap...>
Date:   Sat Sep 28 13:19:07 2013 -0700

    Merge pull request #18 from jaeheung90/dashboard
    
    XML and JSON output of client summary

commit 02aaa341e47fca379ddedbd7f26637ec11a7b689
Author: Jae Park <jae...@gm...>
Date:   Fri Sep 27 16:13:42 2013 -0700

    XML and JSON output of client summary

commit 34dc6c68dac5fd16d23b1c23e55947564ea065af
Author: Jason Heiss <jh...@ap...>
Date:   Sat Aug 17 12:21:49 2013 -0400

    Use a unique pid file for each test server instance
    
    Rails now complains if you ask it to start a new server instance when
    there's already one running using the default pid file.

commit f3a5cb1ff9a825e9575abeff13c23d01a9cb928c
Author: Jason Heiss <jh...@ap...>
Date:   Sat Aug 17 12:12:50 2013 -0400

    Update to rails 3.2.14

commit 5fa12a22a330392556824d0af6f57d660ebe035b
Author: Jason Heiss <jh...@ap...>
Date:   Sat Aug 17 12:12:28 2013 -0400

    Update source to https

commit 66b660dfc3ab8b014aa4be3acece7a882296755f
Merge: 08fa2ab d1ab3a2
Author: Jason Heiss <jh...@ap...>
Date:   Sat Aug 17 08:40:47 2013 -0700

    Merge pull request #17 from stnoonan/master
    
    Update etchserver.conf parsing

commit d1ab3a2142095dc4562c08592a67930dad4a7e58
Author: Sean Timothy Noonan <stn...@ob...>
Date:   Fri Aug 16 23:29:33 2013 -0400

    Update etchserver.conf parsing
    
    No longer depends on regular expressions, should be straightforward
    to extend with additional keys.

commit 08fa2ab5f59be451bd0922bf9ca156d22a037bd7
Merge: 3da17c5 f461379
Author: Jason Heiss <jh...@ap...>
Date:   Tue May 21 07:07:32 2013 -0700

    Merge pull request #14 from jaeheung90/debug_app_views_clients
    
    Bug fix: Handle empty result from oldest results query.
commit 3da17c525b27a25802dc006e6ea7fb9050067eb3
Merge: 0130861 5dd5d30
Author: Jason Heiss <jh...@ap...>
Date:   Tue May 21 07:06:51 2013 -0700

    Merge pull request #15 from jaeheung90/debug_dry-run_guard
    
    Let guard exec run in dry-run

commit 5dd5d304f61495f6a62e41ffb479694b769cd255
Author: Jae Park <jp...@yp...>
Date:   Mon May 20 19:55:35 2013 -0700

    Let guard commands run in dry run
    
    To make dry-run meaningful for configuration commands

commit f461379c823abd05ad1ec2adbf84b6d02d041dde
Author: Jae Park <jp...@yp...>
Date:   Mon May 20 18:35:06 2013 -0700

    Handle empty result from oldest results query.
    
    As clients can have no results in normal circumstances.

commit 01308619f182337ad0eb4e65fb5d646b71712951
Author: Jason Heiss <jh...@ap...>
Date:   Sat Apr 20 12:27:10 2013 -0400

    Use encode! instead of encode

commit 82ede8f76ae2b76fb4b72f233778201605562c75
Author: Jason Heiss <jh...@ap...>
Date:   Tue Nov 6 16:43:45 2012 +0000

    Update debian package to work with ruby 1.9

commit 151c57f57a2cd799d387b175d6cec14df56f2f16
Author: Jason Heiss <jh...@ap...>
Date:   Fri Sep 28 21:28:28 2012 -0400

    Change syntax to be ruby 1.8 compatible
commit 201401161e153a24b5e40e00e5c07d666f723371
Author: Jason Heiss <jh...@ap...>
Date:   Wed Sep 26 06:59:45 2012 -0400

    Make binary encoding changes compatible with 1.8

commit ab714b1d46425226fcde3dd696fd2dc2d4c0b947
Author: Jason Heiss <jh...@ap...>
Date:   Tue Sep 25 14:54:14 2012 -0400

    Use ASCII-8BIT when we handle user file data
    
    User managed files, both originals and generated files, may well not be
    UTF-8 (or whatever ruby uses for the default external encoding on the
    user's system).  So we need to tell ruby that the file data should be
    handled as binary data.

commit d73fd5af54ce7a8e47eef2f8251dcb697370027b
Author: Jason Heiss <jh...@ap...>
Date:   Mon Aug 27 22:24:53 2012 -0400

    Update copyright notices

commit bec77c94517c2c67e9ad0940dfc747f0df847d34
Author: Jason Heiss <jh...@ap...>
Date:   Mon Apr 30 01:44:32 2012 +0000

    Remove the calls to unlock_all_files when the user selects quit in
    interactive mode.  Unlocking will naturally occur as the stack of
    process_file and process_command calls unwinds.  Fixes ticket:21

commit b849c465aa1b7c83b3ab5c2c86bf9292c60a3e14
Author: Jason Heiss <jh...@ap...>
Date:   Mon Apr 30 01:02:58 2012 +0000

    When processing dependencies we need to pay attention to the return
    value from process_file or process_command

commit 6f17ec46a050dd34696a74c3e0a7cf92d4b8bc94
Author: Jason Heiss <jh...@ap...>
Date:   Sat Apr 28 02:00:29 2012 +0000

    Fix up get_user_confirmation so that it behaves properly

commit 84162c746e49627cc2c1579b507179137ebeb95b
Author: Jason Heiss <jh...@ap...>
Date:   Sat Apr 28 01:31:19 2012 +0000

    Run unicorn based on whether the user has enabled it in their Gemfile
    rather than based on a hard-coded constant in the test suite.

commit c855b5084eada2eaaa2236b7c21f739e9f5f5a64
Author: Jason Heiss <jh...@ap...>
Date:   Sat Apr 28 01:26:02 2012 +0000

    Split the tests up into individual test methods

commit 0f9fa4bec8d7af875fe192829a1ecf64f3e2ccb3
Author: Jason Heiss <jh...@ap...>
Date:   Fri Apr 27 02:25:53 2012 +0000

    When processing commands if a guard fails then tell the user which
    command we're going to run to address the problem, similar to what we do
    when processing files.  That way if the user is running in interactive
    mode they know what they're agreeing to have done.

commit c1a03a67c81d13c73f71fd77e46f99ada1f444d0
Author: Jason Heiss <jh...@ap...>
Date:   Fri Apr 27 02:00:29 2012 +0000

    Add test_output_encoding
    
    Do a proper skip in test_output_capture_timeout rather than just
    commenting out the assertion.

commit b9f659b25441c33eeb05a06c8ec7fec176fa03ae
Author: Jason Heiss <jh...@ap...>
Date:   Fri Apr 27 01:59:26 2012 +0000

    Store @results as a hash so that we only capture the last result for
    each individual file.  This eliminates a bunch of empty and misleading
    results for files from handling need_sum or need_orig requests.  We only
    want to capture the last result where the contents of the file are
    handled.
    
    If ruby >= 1.9 then specify encoding options to the pipes used for
    output capturing.  Otherwise we may end up capturing non-UTF-8 output
    and sending it to the server as UTF-8, which causes the server to error
    out.
    
    Minor cleanup in lock_file to eliminate a warning about an unused
    variable.

commit 93cd7d384cfc91155cbfe72906ffa7a8d025bba2
Author: Jason Heiss <jh...@ap...>
Date:   Fri Apr 27 01:50:53 2012 +0000

    Switch some instances of ruby 1.9 hash syntax back to 1.8-compatible
    syntax.  Since Rails still supports 1.8 I won't otherwise force folks to
    1.9 yet.

commit a959a86c60bc9c6bd1ab0b5253b63d993d9add03
Author: Jason Heiss <jh...@ap...>
Date:   Fri Apr 27 01:50:09 2012 +0000

    When passing a params key to Rails for possible storage in the database
    (various find_or_create_by_* calls), dup the key.  This eliminates
    intermittent "RuntimeError: can't modify frozen String" errors I was
    getting from the test suite.  The wisdom of the Internets seems to
    indicate that a hash key (all hash keys?  or just params hash keys?
    it's not clear) can be or always are frozen, and thus dup'ing the key
    fixes the problem.  It's a little strange that it's an intermittent
    problem, but the dup fix seems fairly cheap and harmless.

commit 9d7c7c34939db27ba3b17940a630b9d55b835925
Author: Jason Heiss <jh...@ap...>
Date:   Fri Apr 27 01:44:06 2012 +0000

    Update some instances of old search syntax to ransack syntax.  Plus a
    few other minor updates in etchtest.rb

commit b3d6d0802ec326db6ea214a3bfaa9f84216e88ad
Author: Jason Heiss <jh...@ap...>
Date:   Fri Apr 27 01:42:43 2012 +0000

    Switch some instances of ruby 1.9 hash syntax back to 1.8-compatible
    syntax.  Since Rails still supports 1.8 I won't otherwise force folks to
    1.9 yet.

commit 233ccbf2f547eb44e35cd680c4d5803b23fea824
Author: Jason Heiss <jh...@ap...>
Date:   Fri Apr 27 01:41:04 2012 +0000

    Remove old comment about starting the server as needed, did that a while ago

commit acb4f8a3a8f730add000b2e06a258b3140ef82a8
Author: Jason Heiss <jh...@ap...>
Date:   Thu Apr 26 23:34:09 2012 +0000

    Update now that this is client-specific

commit 786ec2e7f428ab41126e3e4d955290cfba554b66
Author: Jason Heiss <jh...@ap...>
Date:   Thu Apr 26 23:32:47 2012 +0000

    Move the top-level Gemfile I created a while back into the client
    directory now that the server directory has its own proper Gemfile.

commit 92900e26840fa5118c1235c9182c2c717959de7b
Author: Jason Heiss <jh...@ap...>
Date:   Tue Apr 24 19:59:57 2012 +0000

    Wrap the config.xml load step with wrap_exception so that the user gets
    a useful error message.

commit e88450922c875a49a0f8522b7dadc05d919a6e6c
Author: Jason Heiss <jh...@ap...>
Date:   Tue Apr 24 19:52:53 2012 +0000

    In fixing the syntax error in my last commit I ended up setting the
    wrong color.  Fixing that now.

commit bda6ac626f5dd4ca1b1f1453723bcb3a766855b9
Author: Jason Heiss <jh...@ap...>
Date:   Tue Apr 24 17:14:10 2012 +0000

    Comment out the links to login/logout.  There has never been support for
    them, and with some of the recent routing changes there is no longer
    routing to even generate the links.

commit b0db043fb072cce3bc81d778bba4151a746c9496
Author: Jason Heiss <jh...@ap...>
Date:   Tue Apr 24 17:02:42 2012 +0000

    Ignore public/assets, which is where assets end up when you do a
    "rake assets:precompile”

commit 90c7e0fc7448c7279b7c74f1085bd32b0863467b
Author: Jason Heiss <jh...@ap...>
Date:   Tue Apr 24 17:01:12 2012 +0000

    chdir back to / after processing individual files or commands.
    
    We chdir into the directory for the individual file/command before
    processing it so that the user can refer to other files in the same
    directory using relative paths.  In practice it hasn't caused any
    problems to leave the server chdir'd into that directory until it
    processes the next file, but it does generate sporadic errors from the
    test suite when the directory disappears as the repository is updated.
    So to be on the safe and robust side we now chdir back to / after
    processing an individual file or command.

commit 6048a93cf4c3928981c0dd69fd3a7cc551f6963f
Author: Jason Heiss <jh...@ap...>
Date:   Tue Apr 24 16:58:35 2012 +0000

    Fix syntax error in CSS file

Re: [etch-users] [nventory-users] ruby version?

[Jason H. <jh...@ap...>]

On Nov 9, 2012, at 1:50 AM, Kenneth Williams <hap...@gm...> wrote:

> Is there a recommendation on which versions of ruby appear to work best with nventory & etch these days? I see the centos install script uses 1.8.7, is that out of convenience, or another reason? Thanks! 

I haven't really kept up with nventory, but for etch either 1.8 or 1.9 should work.  That said, I do all development with 1.9 nowadays, so it's a better bet if you have a choice.

[etch-users] Etch 4.0.0 released

[Jason H. <jh...@ap...>]

Significant changes:

* Server upgraded to rails 3.2.3 from 2.3.14
* Search functionality in server now uses ransack[1] rather than custom code.
* Graphs on etch server dashboard now use Javascript via flot[2] instead of flash

Minor stuff:
* The client should now handle Unicode encoding properly when running under ruby 1.9 so that if the output captured from operations includes non-UTF8 characters they'll be replaced by something valid rather than causing the server to reject the message.
* Fix interactive mode so that it works again

Notes:

The GettingStarted page on the wiki has been updated to reflect how to get Rails 3 up and running.  Note that Rails 3 requires ruby 1.8.7, 1.9.2 or 1.9.3.  The client should still run under any version of 1.8 or 1.9.

The server search engine change breaks any custom search bookmarks you might have.  If you need help figuring out the new syntax just ask here on the list.

[1]  https://github.com/ernie/ransack
[2]  http://code.google.com/p/flot/

Re: [etch-users] 500 Server Error

[Phillip S. <fu...@gm...>]

On 28 March 2012 16:38, Phillip Smith <fu...@gm...> wrote:
>
> Any ideas?
>


My apologies for the noise, but I found the problem so for the benefit
of future Googlers... The "nodetagger" script was missing from my
config tree. Copied the sample one in there and all is happy in the
world :)

Re: [etch-users] 500 Server Error

[Phillip S. <fu...@gm...>]

On 28 March 2012 16:38, Phillip Smith <fu...@gm...> wrote:
> I'm trying to get an etch server setup, but I'm getting the following error:
>
> svr-etch /usr/local/src/etch-3.20.1/server # etch --dry-run
> --generate-all --server http://localhost:8080
> External node tagger exited with error 127
> 500 "Internal Server Error"


FWIW, running etch in local mode works as expected so I don't think
it's anything to do with the actual etch configuration tree:

svr-etch /etc/etchserver # etch --dry-run --generate-all --local
/etc/etchserver/
Making directory tree /var/etch/orig/etc
Saving temporary copy of original file:  /etc/sudoers ->
/var/etch/orig/etc/sudoers.TMP
Making directory tree /var/etch/orig/etc
...... <snip> .........
Making history directory /var/etch/history/etc/ssh/sshd_config.HISTORY
Updating history log:  /etc/ssh/sshd_config ->
/var/etch/history/etc/ssh/sshd_config.HISTORY/current

[etch-users] 500 Server Error

[Phillip S. <fu...@gm...>]

Hi again folks,

I'm trying to get an etch server setup, but I'm getting the following error:

svr-etch /usr/local/src/etch-3.20.1/server # etch --dry-run
--generate-all --server http://localhost:8080
External node tagger exited with error 127
500 "Internal Server Error"

The server logs appear to indicate nothing useful:

svr-etch /usr/local/src/etch-3.20.1/server # tail log/development.log
  Fact Load (0.7ms)   SELECT * FROM "facts" WHERE ("facts"."key" =
'selinux_enforced' AND "facts"."client_id" = 1) LIMIT 1
  Fact Load (0.7ms)   SELECT * FROM "facts" WHERE ("facts"."key" =
'memorytotal' AND "facts"."client_id" = 1) LIMIT 1
  Fact Load (0.7ms)   SELECT * FROM "facts" WHERE ("facts"."key" =
'architecture' AND "facts"."client_id" = 1) LIMIT 1
  Fact Load (0.7ms)   SELECT * FROM "facts" WHERE ("facts"."key" =
'network_eth0' AND "facts"."client_id" = 1) LIMIT 1
  Fact Load (0.7ms)   SELECT * FROM "facts" WHERE ("facts"."key" =
'kernel' AND "facts"."client_id" = 1) LIMIT 1
  Fact Load (0.7ms)   SELECT * FROM "facts" WHERE ("facts"."key" =
'sshrsakey' AND "facts"."client_id" = 1) LIMIT 1
  Fact Load (0.7ms)   SELECT * FROM "facts" WHERE ("facts"."key" =
'sshdsakey' AND "facts"."client_id" = 1) LIMIT 1
  Fact Load (4.3ms)   SELECT * FROM "facts" WHERE ("facts"."client_id" = 1)
External node tagger exited with error 127
Completed in 369ms (View: 2, DB: 50) | 500 Internal Server Error
[http://localhost/files]

Any ideas?

Thx,
~p

Re: [etch-users] Push Configuration

[Pat O'B. <obr...@gm...>]

This conversation got me fairly interested in remotely triggering etch, so
I wrote an etch daemon that replaces the functionality that the current
"etch_wrapper" script which we run out of cron, and also adds the ability
to remotely trigger etch, using sinatra+webrick.

https://github.com/poblahblahblah/etchd

I'll add a gemspec file sooner or later and probably break out some things
into a config file (/etc/etch/etchd.conf) or some such. I was thinking
about an "etchctl" script which you could user to trigger etch runs across
multiple machines simultaneously.

This could use better exit and status handling, but to trigger on the
command line you can do something like this:

curl -k -d "my_pass=super_secret_password"
https://client.colo.domain.com/run_etch/trunk

this would run etch against the "trunk" tag, if you want to run it against
a specific tag, you would do this:

curl -k -d "my_pass=super_secret_password"
https://client.colo.domain.com/run_etch/tag-2011032601

You will have to generate a password hash and paste it in the etchd.rb,
here is how you would do that using BCrypt in ruby:


-----snip-----
require 'rubygems'
require 'bcrypt'

puts BCrypt::Engine.hash_secret("super_secret_password",
BCrypt::Engine.generate_salt)
-----snip-----

You'll also want to generate a self signed key and cert and put them as
/etc/etch/etchd.crt and /etc/etch/etchd.key - these should probably be
broken out into a config file as well.

Anyway, it's pretty rough, but it seems to work well in the testing I've
done with it, so if there is any interest beyond our own needs I'll develop
it further and eventually push it over to the sourceforge repo.

-pat


On Tue, Mar 27, 2012 at 6:27 AM, Jason Heiss <jh...@ap...> wrote:

>
> On Mar 26, 2012, at 10:08 PM, Phillip Smith wrote:
>
> > That would be perfect IMHO... If I had any skills in Ruby I would have a
> crack at doing this myself. Technically it's not pushing, just provoking
> the client to initiate a pull on demand, but it achieves the same result;
> changes are propagated promptly.
> >
> > I imagine it would introduce a small security implication since the
> daemon would have to be run as root to be able to spawn etch as root, but
> legitimate connections would come from a known source so could be
> firewalled easily.
>
> The daemon would need to run as root (or have NOPASSWD sudo to run etch),
> but if that's all it could do, didn't accept any specific commands or
> options from the network, etc. then the only security implications would be
> denial of service if someone pinged it continuously.  At worst you'd just
> have the resource consumption of etch running continuously on your systems
> if someone triggered it constantly.
>
> But using some sort of shared secret like Pat suggested probably is a good
> idea to reduce the chance of that.
>
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> etch-users mailing list
> etc...@li...
> https://lists.sourceforge.net/lists/listinfo/etch-users
>
>

Re: [etch-users] Push Configuration

[Jason H. <jh...@ap...>]

On Mar 26, 2012, at 10:08 PM, Phillip Smith wrote:

> That would be perfect IMHO... If I had any skills in Ruby I would have a crack at doing this myself. Technically it's not pushing, just provoking the client to initiate a pull on demand, but it achieves the same result; changes are propagated promptly.
> 
> I imagine it would introduce a small security implication since the daemon would have to be run as root to be able to spawn etch as root, but legitimate connections would come from a known source so could be firewalled easily.

The daemon would need to run as root (or have NOPASSWD sudo to run etch), but if that's all it could do, didn't accept any specific commands or options from the network, etc. then the only security implications would be denial of service if someone pinged it continuously.  At worst you'd just have the resource consumption of etch running continuously on your systems if someone triggered it constantly.

But using some sort of shared secret like Pat suggested probably is a good idea to reduce the chance of that.

Re: [etch-users] Push Configuration

[Phillip S. <fu...@gm...>]

On 27 March 2012 12:54, Jason Heiss <jh...@ap...> wrote:

Thanks for your thoughtful reply; nice to get a good response from a dev
instead of a black-and-white "no" :)


> The problem with the first approach is that you're going to beat up your
> etch servers with the heavy polling, most of which is useless (since you
> will be pushing out of cycle updates very infrequently relative to the
> polling frequency).  If your environment is relatively small this may be
> acceptable though.  I can imagine ways to use the existing "tag"
> functionality to make this work.  For example, you could have in cron:
>

Agreed... While this is a (very) small site, I still hate
"hammer-to-crack-nut" solutions ;)


>
> * * * * *   etch --generate-all --tag urgent
> 0 * * * *  etch --generate-all
>
> /etc/etchserver/urgent could normally be an empty directory, but if you
> had something urgent to push you could populate that directory with an
> updated config repository.  Once all the clients had picked up the change
> and the change had propagated into your normal tags then you could clean
> the urgent directory out again.  You could get fancier with adding a
> timestamp to the tag (i.e. urgent-`date +%Y%m%d%H%M`) and then you wouldn't
> have to bother with cleaning out the directory.
>

I will have to read up more on tags to fully understand this suggestion,
but it still has load implications for the server, and
"hammer-to-crack-nut" feel to it.



> I've thought about building a special daemon that just listens on a port
> and if it gets a connection it runs etch.  That would probably be faster
> than using ssh, but in the end I've always decided it's not worth the
> effort.  It would be pretty easy to do though if you wanted that approach.
>

That would be perfect IMHO... If I had any skills in Ruby I would have a
crack at doing this myself. Technically it's not pushing, just provoking
the client to initiate a pull on demand, but it achieves the same result;
changes are propagated promptly.

I imagine it would introduce a small security implication since the daemon
would have to be run as root to be able to spawn etch as root, but
legitimate connections would come from a known source so could be
firewalled easily.

Re: [etch-users] Push Configuration

[Jason H. <jh...@ap...>]

On Mar 26, 2012, at 7:35 PM, Phillip Smith wrote:

> Just found etch the other day and it looks great... I've had puppet on my "todo" list for a while, but etch is simpler but still sufficient for what I want.

Just my personal opinion, but if you decide etch isn't your thing you should look at chef rather than puppet.  It's still too complicated, but more understandable than puppet.

> One query though, there appears to be no way to push config to a client (ie, the client has to connect to the server and pull it's config).
> 
> Are there any workarounds or plans to implement a push system so when a config is changed on the server, it pushes it to the client immediately without having to either a) login to the client and pull it or b) wait for a scheduled cron pull or similar?
> 
> It's not so bad for 1 client, but if you update a file that affects multiple clients, it would be (number_of_clients) times easier to be able to push it from the one location than having to login to each client :)


This comes up every once in a while, but so far I don't think anyone has found it to be an issue in practice.  Usually if you really need something to go out quickly you just need in on a manageable number of hosts (less than 100) and can use a for loop and ssh to the boxes to run etch out of cycle.  In really large environments (1000s or 10000s of hosts) the risk of pushing something quickly to all hosts is just too great, so you learn to accept the delay.

But for sake of argument I can talk a bit about how you might approach this if you really want it.  From a purely technical standpoint you have two major approaches:

* Run something in cron on the clients more frequently (every minute or every 5 minutes for example) that checks for special, out-of-cycle update requests.
* Poke the clients to run etch out of cycle.

The problem with the first approach is that you're going to beat up your etch servers with the heavy polling, most of which is useless (since you will be pushing out of cycle updates very infrequently relative to the polling frequency).  If your environment is relatively small this may be acceptable though.  I can imagine ways to use the existing "tag" functionality to make this work.  For example, you could have in cron:

* * * * *   etch --generate-all --tag urgent
0 * * * *  etch --generate-all

/etc/etchserver/urgent could normally be an empty directory, but if you had something urgent to push you could populate that directory with an updated config repository.  Once all the clients had picked up the change and the change had propagated into your normal tags then you could clean the urgent directory out again.  You could get fancier with adding a timestamp to the tag (i.e. urgent-`date +%Y%m%d%H%M`) and then you wouldn't have to bother with cleaning out the directory.

The problem with the second approach (as you mentioned) is that for a non-trivial number of hosts the time to run a for loop over all of them is probably longer than you'd like, requires the ability to ssh in and run something as root (which is often locked down for security reasons), etc.  I've thought about building a special daemon that just listens on a port and if it gets a connection it runs etch.  That would probably be faster than using ssh, but in the end I've always decided it's not worth the effort.  It would be pretty easy to do though if you wanted that approach.

[etch-users] Push Configuration

[Phillip S. <fu...@gm...>]

Just found etch the other day and it looks great... I've had puppet on my
"todo" list for a while, but etch is simpler but still sufficient for what
I want.

One query though, there appears to be no way to push config to a client
(ie, the client has to connect to the server and pull it's config).

Are there any workarounds or plans to implement a push system so when a
config is changed on the server, it pushes it to the client immediately
without having to either a) login to the client and pull it or b) wait for
a scheduled cron pull or similar?

It's not so bad for 1 client, but if you update a file that affects
multiple clients, it would be (number_of_clients) times easier to be able
to push it from the one location than having to login to each client :)

Cheers,
~p

[etch-users] Etch 3.20.1 released

[Jason H. <jh...@ap...>]

A small patch release to address a few minor things:

* The warning that etch is defaulting to UID/GID 0 because a user or group was not found is now shown only in debug mode.  Change contributed by Marjorie Saltman.
* Fixed a typo in the client Rakefile

[etch-users] Etch 3.20.0 released

[Jason H. <jh...@ap...>]

Significant changes:

* Etch is now compatible with ruby 1.9
* Nokogiri is now the default XML parser used by the server.  Libxml and REXML are still supported as alternates.
* Switch server to rails 2.3.14 (from 2.3.11)

Other changes:

* Make configdir and vardir configurable via command-line options.  Suggestion from Nate Johnston.  See etch(8) man page for details.
* Add additional debugging statement about the server URL in use.  Suggestion from Nate Johnston.
* Directory structure under client/ rearranged to follow the typical layout of a Ruby project (bin/, lib/, etc. directories)
* The client packaging metafiles (the spec file for RPMs and the control file for debs) now list cpio as a dependency.  It has always been a dependency, it just got left out of the list.
* The DTD files in the demo and samples directories have been updated.
* Fixed a bug when the REXML parser is in use.
* The location of the server's debug log (used when clients request debug level logging) can now be configured via the etchdebuglog setting in etchserver.conf.  Change contributed by Pat O'Brien.
* Facter facts are explicitly converted to strings when running in local mode to ensure the same behavior as client/server mode (where they are implicitly converted to strings by virtue of being passed over HTTP).  Change contributed by Pat O'Brien.
* Added a bundler Gemfile to the svn repository to make it easier for new developers to get the necessary gems installed.  Just run "bundle install"
* Removed -w from the ruby command line options in all the executables.  It is too hard to eliminate warnings from
libraries (mostly facter), which makes these programs noisy under ruby 1.9.
* Increase output capture timeout in interactive mode to allow the user more time to respond to interactive questions.

Re: [etch-users] couple of questions regarding etch

[Jason H. <jh...@ap...>]

On Mar 16, 2012, at 3:20 PM, George MacLachlan wrote:

> 1)    How widely distributed is this product and what is the current activity (update frequency, number
> of fixes per update)?

I know of two medium sized businesses running the current major version of etch, each managing somewhere in the 1000-5000 server range.  I used an older version of etch to manage over 20,000 servers in another company.  I also happen to run it on my two personal servers.

So it has been well tested in a few relatively large environments, but does not have the hundreds or thousands of small environments running it like some of the competing products (chef, puppet, cfengine).

Releases are fairly infrequent.  There were 2 in 2011, 3 (plus a minor release) in 2010.  You can look back through the archives of this mailing list to find more details of what was in each release.  The low rate of releases is mostly because etch is stable and seems to do what the folks using it need it to do.  We're overdue for a new release, there are some minor bug fixes and enhancements in subversion that should get pushed out.

You can view all the open tickets at the URL below.  As you can see there are only two defect tickets open and both are minor issues that haven't bugged anyone enough to fix.  The rest are feature enhancements of various sorts that might be nice to have but again not enough that anyone has done them yet.

http://sourceforge.net/apps/trac/etch/report/1

>  2)    What is the level of support  ?

This mailing list is the primary and only support mechanism, short of hiring one of the developers or hiring your own developer.  If you have someone who knows or can learn ruby it wouldn't be that hard to dig into the etch codebase.  It's not that big, certainly much smaller than something like puppet or chef.

>   3)    Has anyone taken this implementation and migrated it to an enterprise version that can be
> licensed for a fee?

No.  I did that for the previous major release some years ago and there wasn't enough business to make it worthwhile.

>  4)    Are there any other software dependencies besides Ruby and Facter Library?

The only other thing, which is usually installed by default, is the cpio command.  And the client can be run in --local mode for initial setup, so that's all you need to get started.

The server has some more dependencies as listed on the GettingStarted wiki page.

[etch-users] couple of questions regarding etch

[George M. <gfm...@gm...>]

My organization is looking for an open source Configuration Management
product for a Linux

Platforms project (approximately 80 servers) and have started evaluating
several candidates, one

of which is Etch.  The existing documentation is quite comprehensive and
answers a lot of my

questions.  However, there are a few items I would appreciate your help
with:



1)    How widely distributed is this product and what is the current
activity (update frequency, number

of fixes per update)?



2)    What is the level of support  ?



3)    Has anyone taken this implementation and migrated it to an enterprise
version that can be

licensed for a fee?



4)    Are there any other software dependencies besides Ruby and Facter
Library?



Thank you for your help.



Regards,





George MacLachlan

tel: (732) 420-5587

Cell: (732) 208-8258

Re: [etch-users] Getting Started on Debian Squeeze

[Jason H. <jh...@ap...>]

Known to work on Debian Squeeze?  Not that I know of.  But it should work.

String#oct certainly does exist in ruby 1.8.  In fact I'd hazard that it's been in ruby since the earliest days since it's a pretty direct carryover from perl.

jheiss@sleet:~/projects/etch/trunk> irb
1.8.7 :001 > RUBY_VERSION
 => "1.8.7" 
1.8.7 :002 > '10'.oct
 => 8 

I just ran the etch test suite under ruby 1.8.7 (on a CentOS 5 box) and it ran fine.  Did you see an error related to String#oct?

With the error you posted below it's not clear what went wrong.  The warning is just that, nothing fatal.  I'm not sure where the "No such file or directory" came from.  I suggest running with --debug and see if that helps, or send us the output and maybe we'll spot something.

Jason

On Feb 28, 2012, at 7:07 AM, Alex Young wrote:

> Hi there,
> 
> Is etch-3.19.0 known to work on Debian Squeeze?  I ask because the 
> Getting Started instructions seem a little broken.
> 
> It would seem that ruby1.8 is no longer supported - String#oct doesn't 
> exist in 1.8, for instance.  This means that on Squeeze, we need to use 
> the ruby1.9.1 package.  `apt-get install facter` only installs the 
> library for 1.8, which means we have to use rubygems.  Using 
> rubygems-1.8.7.tgz from rubygems.org, I can install facter-1.6.5 and run:
> 
>   $ ./etch --generate-all --interactive --local ../etchserver-demo
> 
> but it spews a couple of screens'-worth of warnings before quitting with:
> 
>   W, [2012-02-28T12:00:19.151350 #24223]  WARN -- : No entry found for 
> node <my-hostname> in nodes.xml
>   No such file or directory
> 
> and a failure exit code.  I don't know if that's a sign that I've missed 
> a step somewhere, or if that's actually intended behaviour, or what.  No 
> files get created in /tmp, although /var/etch does get built.
> 
> Any guidance gratefully received.  Thanks,
> -- 
> Alex
> 
> ------------------------------------------------------------------------------
> Keep Your Developer Skills Current with LearnDevNow!
> The most comprehensive online learning library for Microsoft developers
> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
> Metro Style Apps, more. Free future releases when you subscribe now!
> http://p.sf.net/sfu/learndevnow-d2d
> _______________________________________________
> etch-users mailing list
> etc...@li...
> https://lists.sourceforge.net/lists/listinfo/etch-users

[etch-users] Getting Started on Debian Squeeze

[Alex Y. <al...@by...>]

Hi there,

Is etch-3.19.0 known to work on Debian Squeeze?  I ask because the 
Getting Started instructions seem a little broken.

It would seem that ruby1.8 is no longer supported - String#oct doesn't 
exist in 1.8, for instance.  This means that on Squeeze, we need to use 
the ruby1.9.1 package.  `apt-get install facter` only installs the 
library for 1.8, which means we have to use rubygems.  Using 
rubygems-1.8.7.tgz from rubygems.org, I can install facter-1.6.5 and run:

   $ ./etch --generate-all --interactive --local ../etchserver-demo

but it spews a couple of screens'-worth of warnings before quitting with:

   W, [2012-02-28T12:00:19.151350 #24223]  WARN -- : No entry found for 
node <my-hostname> in nodes.xml
   No such file or directory

and a failure exit code.  I don't know if that's a sign that I've missed 
a step somewhere, or if that's actually intended behaviour, or what.  No 
files get created in /tmp, although /var/etch does get built.

Any guidance gratefully received.  Thanks,
-- 
Alex

Re: [etch-users] Request: etchdebug.log to configurable destination

[Johnston, N. <Nat...@ca...>]

Thanks very much!  I will try it out this week.

--N.

On Aug 15, 2011, at 5:51 PM, Pat O'Brien wrote:

Hey Nathaniel,

I committed the necessary changes to trunk so you can specify etchdebuglog=/path/to/wherever in your etchserver.conf file

-pat


On Sun, Aug 7, 2011 at 10:19 PM, Pat O'Brien <obr...@gm...<mailto:obr...@gm...>> wrote:
I've never bothered to try this so I don't know if it's configurable or not. If it's not open a bug ticket and I'll take a look at it when I get some spare time this week.

Just as a side note: my personal preference is to not have system logs end up in /var/log, but to each their own :)

-pat



On Wed, Aug 3, 2011 at 3:27 PM, Johnston, Nathaniel <Nat...@ca...<mailto:Nat...@ca...>> wrote:
Etch Mensches,

I am not sure how best to do this, but I am trying to move my Etch logs over to /var along with the other logfiles I have.  I noticed that on line 205 of lib/etchserver.rb the etchdebug.log file is specified as:

@dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))

Is there a way that this could leverage the value of config.log_path/paths.log or another configurable parameter so that we could override the location of this file?

--N.



------------------------------------------------------------------------------
BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
_______________________________________________
etch-users mailing list
etc...@li...<mailto:etc...@li...>
https://lists.sourceforge.net/lists/listinfo/etch-users

Re: [etch-users] Request: etchdebug.log to configurable destination

[Pat O'B. <obr...@gm...>]

Hey Nathaniel,

I committed the necessary changes to trunk so you can specify
etchdebuglog=/path/to/wherever in your etchserver.conf file

-pat


On Sun, Aug 7, 2011 at 10:19 PM, Pat O'Brien <obr...@gm...>wrote:

> I've never bothered to try this so I don't know if it's configurable or
> not. If it's not open a bug ticket and I'll take a look at it when I get
> some spare time this week.
>
> Just as a side note: my personal preference is to not have system logs end
> up in /var/log, but to each their own :)
>
> -pat
>
>
>
> On Wed, Aug 3, 2011 at 3:27 PM, Johnston, Nathaniel <
> Nat...@ca...> wrote:
>
>> Etch Mensches,
>>
>> I am not sure how best to do this, but I am trying to move my Etch logs
>> over to /var along with the other logfiles I have.  I noticed that on line
>> 205 of lib/etchserver.rb the etchdebug.log file is specified as:
>>
>> @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log',
>> 'etchdebug.log'))
>>
>> Is there a way that this could leverage the value of
>> config.log_path/paths.log or another configurable parameter so that we could
>> override the location of this file?
>>
>> --N.
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
>> The must-attend event for mobile developers. Connect with experts.
>> Get tools for creating Super Apps. See the latest technologies.
>> Sessions, hands-on labs, demos & much more. Register early & save!
>> http://p.sf.net/sfu/rim-blackberry-1
>> _______________________________________________
>> etch-users mailing list
>> etc...@li...
>> https://lists.sourceforge.net/lists/listinfo/etch-users
>>
>
>

Re: [etch-users] Request: etchdebug.log to configurable destination

[Jason H. <jh...@ap...>]

(Moving this to etch-devel, bcc'ing out etch-users)

It looks like you're adopting a different syntax for your "etchdebuglog" from the accepted syntax for the existing config file entries.  E.g. the line for matching the "auth_enabled" option is:

if line =~ /^\s*auth_enabled\s*=\s*(.*)/

Whereas you have:

if line =~ /^etchdebuglog:/

I think we want to retain a common syntax.

For the later half of your changes I personally find ternary operator syntax hard to read at a glance.  My wife tells me that's because I'm not a real developer, so shrug, but I have to stare at this a while to figure out what is going on:

debug ? @dlogger.level = Logger::DEBUG : @dlogger.level = Logger::INFO

Whereas this is obvious to me at a glance:

if debug
  @dlogger.level = Logger::DEBUG
else
  @dlogger.level = Logger::INFO
end

Maybe I'm special.

Jason

On Aug 8, 2011, at 11:40 PM, Pat O'Brien wrote:

> sounds good - how does this look:
> 
> ---------------------------->%------------------------------
> wordup:etch pobrien$ svn diff
> Index: server.rb
> ===================================================================
> --- server.rb   (revision 298)
> +++ server.rb   (working copy)
> @@ -24,14 +24,16 @@
>      end
>      @@configbase
>    end
> -  
> +
>    @@auth_enabled = nil
>    @@auth_deny_new_clients = nil
> +  @@etchdebuglog = nil  
>    def self.read_config_file
>      config_file = File.join(configbase, 'etchserver.conf')
>      if File.exist?(config_file)
>        auth_enabled = false
>        auth_deny_new_clients = false
> +      etchdebuglog = nil
>        IO.foreach(config_file) do |line|
>          # Skip blank lines and comments
>          next if line =~ /^\s*$/
> @@ -47,7 +49,11 @@
>              auth_deny_new_clients = true
>            end
>          end
> +        if line =~ /^etchdebuglog:/
> +          etchdebuglog = line.split.pop
> +        end
>        end
> +      @@etchdebuglog = etchdebuglog
>        @@auth_enabled = auth_enabled
>        @@auth_deny_new_clients = auth_deny_new_clients
>      end
> @@ -203,12 +209,11 @@
>      @facts = facts
>      @tag = tag
>      @debug = debug
> -    @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
> -    if debug
> -      @dlogger.level = Logger::DEBUG
> -    else
> -      @dlogger.level = Logger::INFO
> -    end
> +
> +    @@etchdebuglog ? @dlogger = Logger.new(@@etchdebuglog) :  @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
> +
> +    debug ? @dlogger.level = Logger::DEBUG : @dlogger.level = Logger::INFO
> +
>      @fqdn = @facts['fqdn']
>  
>      if !@fqdn
> ---------------------------->%------------------------------
> 
> -pat
> 
> On Mon, Aug 8, 2011 at 5:54 PM, Jason Heiss <jh...@ap...> wrote:
> My thoughts:
> 
> - Don't duplicate the logic of the self.read_config_file method, add your option into the existing config file parsing in that method.
> 
> - Rather than an option for a log directory (your "log_dir"), I'd vote for an option that specifies the full path for this particular log file.  I.e. "debuglog=/path/to/something" or similar.  I think giving a log directory option is misleading, as most of the "etch server" logs are actually emitted by Rails and the Ruby app server (unicorn or the like) and as such would still end up in <rails root>/log even if the user specified a different log directory in etchserver.conf.  The etch debug log is the only log file emitted by the etch server code itself.
> 
> Jason
> 
> On Aug 7, 2011, at 11:45 PM, Pat O'Brien wrote:
> 
>> Well, I worked out a fix for this, although I am unsure if it's a good fix or not.
>> 
>> First, here is the diff:
>> 
>> ------------------------------>%------------------------------
>> wordup:server pobrien$ svn diff
>> Index: lib/etch/server.rb
>> ===================================================================
>> --- lib/etch/server.rb  (revision 298)
>> +++ lib/etch/server.rb  (working copy)
>> @@ -203,12 +203,24 @@
>>      @facts = facts
>>      @tag = tag
>>      @debug = debug
>> -    @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
>> -    if debug
>> -      @dlogger.level = Logger::DEBUG
>> -    else
>> -      @dlogger.level = Logger::INFO
>> +    @log_dir = nil
>> +    config_file = File.join(Etch::Server.configbase, 'etchserver.conf')
>> +    if File.exist?(config_file)
>> +      IO.foreach(config_file) do |line|
>> +        # Skip blank lines and comments
>> +         next if line =~ /^\s*$/
>> +         next if line =~ /^\s*#/
>> +        line.chomp
>> +        if line =~ /^log_dir:/
>> +          @log_dir = line.split.pop
>> +        end
>> +      end
>>      end
>> +
>> +    @log_dir ? @dlogger = Logger.new(File.join(@log_dir, 'etchdebug.log')) :  @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
>> +
>> +    debug ? @dlogger.level = Logger::DEBUG : @dlogger.level = Logger::INFO
>> +
>>      @fqdn = @facts['fqdn']
>>  
>>      if !@fqdn
>> ------------------------------>%------------------------------
>> 
>> Here are my problems with the change: the first is that I'm not sure if this config should go into <etchbase>/etchserver.conf or if it should be within the RAILS_ROOT/config/ directory. This feels like something that should be configured along with the rails app rather than in /etc/etchserver, but from what I understand (and I am in no way, shape or form versed in rails) in rails 2.x.x there is no generic config file to handle something like this. There is a railscast (http://railscasts.com/episodes/85-yaml-configuration-file) that touches upon this, and they basically recommend RAILS_ROOT/config/config.yml along with some other necessary changes to actually have the config file loaded.
>> 
>> Second, it looks like the <etchbase>/etchserver.conf file is currently only used to define two things:  @@auth_enabled and @@auth_deny_new_clients = nil, and the way these are loaded is by parsing the config file the same way I parse it for the log_dir setting. I would prefer etchserver.conf file to be a YAML file and we could load it that way, but I haven't looked too deeply as to what else this file is used for so I don't really want to make any recommendations on how to change it.
>> 
>> Third, I haven't looked hard enough to see whether or not anything else references a log directory of any kind and because of my two issues above I won't bother until there's a bit more discussion on it.
>> 
>> Other than that the change works fine and behaves as you would expect it to, your etchserver.conf file would look like this:
>> 
>> log_dir: /whatever/directory/your/heart/desires
>> 
>> -pat
>> 
>> 
>> 
>> 
>> On Sun, Aug 7, 2011 at 10:19 PM, Pat O'Brien <obr...@gm...> wrote:
>> I've never bothered to try this so I don't know if it's configurable or not. If it's not open a bug ticket and I'll take a look at it when I get some spare time this week.
>> 
>> Just as a side note: my personal preference is to not have system logs end up in /var/log, but to each their own :)
>> 
>> -pat
>> 
>> 
>> 
>> On Wed, Aug 3, 2011 at 3:27 PM, Johnston, Nathaniel <Nat...@ca...> wrote:
>> Etch Mensches,
>> 
>> I am not sure how best to do this, but I am trying to move my Etch logs over to /var along with the other logfiles I have.  I noticed that on line 205 of lib/etchserver.rb the etchdebug.log file is specified as:
>> 
>> @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
>> 
>> Is there a way that this could leverage the value of config.log_path/paths.log or another configurable parameter so that we could override the location of this file?
>> 
>> --N.
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
>> The must-attend event for mobile developers. Connect with experts.
>> Get tools for creating Super Apps. See the latest technologies.
>> Sessions, hands-on labs, demos & much more. Register early & save!
>> http://p.sf.net/sfu/rim-blackberry-1
>> _______________________________________________
>> etch-users mailing list
>> etc...@li...
>> https://lists.sourceforge.net/lists/listinfo/etch-users
>> 
>> 
>> ------------------------------------------------------------------------------
>> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
>> The must-attend event for mobile developers. Connect with experts. 
>> Get tools for creating Super Apps. See the latest technologies.
>> Sessions, hands-on labs, demos & much more. Register early & save!
>> http://p.sf.net/sfu/rim-blackberry-1_______________________________________________
>> etch-users mailing list
>> etc...@li...
>> https://lists.sourceforge.net/lists/listinfo/etch-users
> 
> 

Re: [etch-users] Request: etchdebug.log to configurable destination

[Pat O'B. <obr...@gm...>]

sounds good - how does this look:

---------------------------->%------------------------------
wordup:etch pobrien$ svn diff
Index: server.rb
===================================================================
--- server.rb   (revision 298)
+++ server.rb   (working copy)
@@ -24,14 +24,16 @@
     end
     @@configbase
   end
-
+
   @@auth_enabled = nil
   @@auth_deny_new_clients = nil
+  @@etchdebuglog = nil
   def self.read_config_file
     config_file = File.join(configbase, 'etchserver.conf')
     if File.exist?(config_file)
       auth_enabled = false
       auth_deny_new_clients = false
+      etchdebuglog = nil
       IO.foreach(config_file) do |line|
         # Skip blank lines and comments
         next if line =~ /^\s*$/
@@ -47,7 +49,11 @@
             auth_deny_new_clients = true
           end
         end
+        if line =~ /^etchdebuglog:/
+          etchdebuglog = line.split.pop
+        end
       end
+      @@etchdebuglog = etchdebuglog
       @@auth_enabled = auth_enabled
       @@auth_deny_new_clients = auth_deny_new_clients
     end
@@ -203,12 +209,11 @@
     @facts = facts
     @tag = tag
     @debug = debug
-    @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log',
'etchdebug.log'))
-    if debug
-      @dlogger.level = Logger::DEBUG
-    else
-      @dlogger.level = Logger::INFO
-    end
+
+    @@etchdebuglog ? @dlogger = Logger.new(@@etchdebuglog) :  @dlogger =
Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
+
+    debug ? @dlogger.level = Logger::DEBUG : @dlogger.level = Logger::INFO
+
     @fqdn = @facts['fqdn']

     if !@fqdn
---------------------------->%------------------------------

-pat

On Mon, Aug 8, 2011 at 5:54 PM, Jason Heiss <jh...@ap...> wrote:

> My thoughts:
>
> - Don't duplicate the logic of the self.read_config_file method, add your
> option into the existing config file parsing in that method.
>
> - Rather than an option for a log directory (your "log_dir"), I'd vote for
> an option that specifies the full path for this particular log file.  I.e.
> "debuglog=/path/to/something" or similar.  I think giving a log directory
> option is misleading, as most of the "etch server" logs are actually emitted
> by Rails and the Ruby app server (unicorn or the like) and as such would
> still end up in <rails root>/log even if the user specified a different log
> directory in etchserver.conf.  The etch debug log is the only log file
> emitted by the etch server code itself.
>
> Jason
>
> On Aug 7, 2011, at 11:45 PM, Pat O'Brien wrote:
>
> Well, I worked out a fix for this, although I am unsure if it's a good fix
> or not.
>
> First, here is the diff:
>
> ------------------------------>%------------------------------
> wordup:server pobrien$ svn diff
> Index: lib/etch/server.rb
> ===================================================================
> --- lib/etch/server.rb  (revision 298)
> +++ lib/etch/server.rb  (working copy)
> @@ -203,12 +203,24 @@
>      @facts = facts
>      @tag = tag
>      @debug = debug
> -    @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log',
> 'etchdebug.log'))
> -    if debug
> -      @dlogger.level = Logger::DEBUG
> -    else
> -      @dlogger.level = Logger::INFO
> +    @log_dir = nil
> +    config_file = File.join(Etch::Server.configbase, 'etchserver.conf')
> +    if File.exist?(config_file)
> +      IO.foreach(config_file) do |line|
> +        # Skip blank lines and comments
> +         next if line =~ /^\s*$/
> +         next if line =~ /^\s*#/
> +        line.chomp
> +        if line =~ /^log_dir:/
> +          @log_dir = line.split.pop
> +        end
> +      end
>      end
> +
> +    @log_dir ? @dlogger = Logger.new(File.join(@log_dir, 'etchdebug.log'))
> :  @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log',
> 'etchdebug.log'))
> +
> +    debug ? @dlogger.level = Logger::DEBUG : @dlogger.level = Logger::INFO
> +
>      @fqdn = @facts['fqdn']
>
>      if !@fqdn
> ------------------------------>%------------------------------
>
> Here are my problems with the change: the first is that I'm not sure if
> this config should go into <etchbase>/etchserver.conf or if it should be
> within the RAILS_ROOT/config/ directory. This feels like something that
> should be configured along with the rails app rather than in
> /etc/etchserver, but from what I understand (and I am in no way, shape or
> form versed in rails) in rails 2.x.x there is no generic config file to
> handle something like this. There is a railscast (
> http://railscasts.com/episodes/85-yaml-configuration-file) that touches
> upon this, and they basically recommend RAILS_ROOT/config/config.yml along
> with some other necessary changes to actually have the config file loaded.
>
> Second, it looks like the <etchbase>/etchserver.conf file is currently only
> used to define two things:  @@auth_enabled and @@auth_deny_new_clients =
> nil, and the way these are loaded is by parsing the config file the same way
> I parse it for the log_dir setting. I would prefer etchserver.conf file to
> be a YAML file and we could load it that way, but I haven't looked too
> deeply as to what else this file is used for so I don't really want to make
> any recommendations on how to change it.
>
> Third, I haven't looked hard enough to see whether or not anything else
> references a log directory of any kind and because of my two issues above I
> won't bother until there's a bit more discussion on it.
>
> Other than that the change works fine and behaves as you would expect it
> to, your etchserver.conf file would look like this:
>
> log_dir: /whatever/directory/your/heart/desires
>
> -pat
>
>
>
>
> On Sun, Aug 7, 2011 at 10:19 PM, Pat O'Brien <obr...@gm...>wrote:
>
>> I've never bothered to try this so I don't know if it's configurable or
>> not. If it's not open a bug ticket and I'll take a look at it when I get
>> some spare time this week.
>>
>> Just as a side note: my personal preference is to not have system logs end
>> up in /var/log, but to each their own :)
>>
>> -pat
>>
>>
>>
>> On Wed, Aug 3, 2011 at 3:27 PM, Johnston, Nathaniel <
>> Nat...@ca...> wrote:
>>
>>> Etch Mensches,
>>>
>>> I am not sure how best to do this, but I am trying to move my Etch logs
>>> over to /var along with the other logfiles I have.  I noticed that on line
>>> 205 of lib/etchserver.rb the etchdebug.log file is specified as:
>>>
>>> @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log',
>>> 'etchdebug.log'))
>>>
>>> Is there a way that this could leverage the value of
>>> config.log_path/paths.log or another configurable parameter so that we could
>>> override the location of this file?
>>>
>>> --N.
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
>>> The must-attend event for mobile developers. Connect with experts.
>>> Get tools for creating Super Apps. See the latest technologies.
>>> Sessions, hands-on labs, demos & much more. Register early & save!
>>> http://p.sf.net/sfu/rim-blackberry-1
>>> _______________________________________________
>>> etch-users mailing list
>>> etc...@li...
>>> https://lists.sourceforge.net/lists/listinfo/etch-users
>>>
>>
>>
>
> ------------------------------------------------------------------------------
> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
> The must-attend event for mobile developers. Connect with experts.
> Get tools for creating Super Apps. See the latest technologies.
> Sessions, hands-on labs, demos & much more. Register early & save!
>
> http://p.sf.net/sfu/rim-blackberry-1_______________________________________________
> etch-users mailing list
> etc...@li...
> https://lists.sourceforge.net/lists/listinfo/etch-users
>
>
>

Re: [etch-users] Request: etchdebug.log to configurable destination

[Jason H. <jh...@ap...>]

My thoughts:

- Don't duplicate the logic of the self.read_config_file method, add your option into the existing config file parsing in that method.

- Rather than an option for a log directory (your "log_dir"), I'd vote for an option that specifies the full path for this particular log file.  I.e. "debuglog=/path/to/something" or similar.  I think giving a log directory option is misleading, as most of the "etch server" logs are actually emitted by Rails and the Ruby app server (unicorn or the like) and as such would still end up in <rails root>/log even if the user specified a different log directory in etchserver.conf.  The etch debug log is the only log file emitted by the etch server code itself.

Jason

On Aug 7, 2011, at 11:45 PM, Pat O'Brien wrote:

> Well, I worked out a fix for this, although I am unsure if it's a good fix or not.
> 
> First, here is the diff:
> 
> ------------------------------>%------------------------------
> wordup:server pobrien$ svn diff
> Index: lib/etch/server.rb
> ===================================================================
> --- lib/etch/server.rb  (revision 298)
> +++ lib/etch/server.rb  (working copy)
> @@ -203,12 +203,24 @@
>      @facts = facts
>      @tag = tag
>      @debug = debug
> -    @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
> -    if debug
> -      @dlogger.level = Logger::DEBUG
> -    else
> -      @dlogger.level = Logger::INFO
> +    @log_dir = nil
> +    config_file = File.join(Etch::Server.configbase, 'etchserver.conf')
> +    if File.exist?(config_file)
> +      IO.foreach(config_file) do |line|
> +        # Skip blank lines and comments
> +         next if line =~ /^\s*$/
> +         next if line =~ /^\s*#/
> +        line.chomp
> +        if line =~ /^log_dir:/
> +          @log_dir = line.split.pop
> +        end
> +      end
>      end
> +
> +    @log_dir ? @dlogger = Logger.new(File.join(@log_dir, 'etchdebug.log')) :  @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
> +
> +    debug ? @dlogger.level = Logger::DEBUG : @dlogger.level = Logger::INFO
> +
>      @fqdn = @facts['fqdn']
>  
>      if !@fqdn
> ------------------------------>%------------------------------
> 
> Here are my problems with the change: the first is that I'm not sure if this config should go into <etchbase>/etchserver.conf or if it should be within the RAILS_ROOT/config/ directory. This feels like something that should be configured along with the rails app rather than in /etc/etchserver, but from what I understand (and I am in no way, shape or form versed in rails) in rails 2.x.x there is no generic config file to handle something like this. There is a railscast (http://railscasts.com/episodes/85-yaml-configuration-file) that touches upon this, and they basically recommend RAILS_ROOT/config/config.yml along with some other necessary changes to actually have the config file loaded.
> 
> Second, it looks like the <etchbase>/etchserver.conf file is currently only used to define two things:  @@auth_enabled and @@auth_deny_new_clients = nil, and the way these are loaded is by parsing the config file the same way I parse it for the log_dir setting. I would prefer etchserver.conf file to be a YAML file and we could load it that way, but I haven't looked too deeply as to what else this file is used for so I don't really want to make any recommendations on how to change it.
> 
> Third, I haven't looked hard enough to see whether or not anything else references a log directory of any kind and because of my two issues above I won't bother until there's a bit more discussion on it.
> 
> Other than that the change works fine and behaves as you would expect it to, your etchserver.conf file would look like this:
> 
> log_dir: /whatever/directory/your/heart/desires
> 
> -pat
> 
> 
> 
> 
> On Sun, Aug 7, 2011 at 10:19 PM, Pat O'Brien <obr...@gm...> wrote:
> I've never bothered to try this so I don't know if it's configurable or not. If it's not open a bug ticket and I'll take a look at it when I get some spare time this week.
> 
> Just as a side note: my personal preference is to not have system logs end up in /var/log, but to each their own :)
> 
> -pat
> 
> 
> 
> On Wed, Aug 3, 2011 at 3:27 PM, Johnston, Nathaniel <Nat...@ca...> wrote:
> Etch Mensches,
> 
> I am not sure how best to do this, but I am trying to move my Etch logs over to /var along with the other logfiles I have.  I noticed that on line 205 of lib/etchserver.rb the etchdebug.log file is specified as:
> 
> @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log', 'etchdebug.log'))
> 
> Is there a way that this could leverage the value of config.log_path/paths.log or another configurable parameter so that we could override the location of this file?
> 
> --N.
> 
> 
> 
> ------------------------------------------------------------------------------
> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
> The must-attend event for mobile developers. Connect with experts.
> Get tools for creating Super Apps. See the latest technologies.
> Sessions, hands-on labs, demos & much more. Register early & save!
> http://p.sf.net/sfu/rim-blackberry-1
> _______________________________________________
> etch-users mailing list
> etc...@li...
> https://lists.sourceforge.net/lists/listinfo/etch-users
> 
> 
> ------------------------------------------------------------------------------
> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
> The must-attend event for mobile developers. Connect with experts. 
> Get tools for creating Super Apps. See the latest technologies.
> Sessions, hands-on labs, demos & much more. Register early & save!
> http://p.sf.net/sfu/rim-blackberry-1_______________________________________________
> etch-users mailing list
> etc...@li...
> https://lists.sourceforge.net/lists/listinfo/etch-users

Re: [etch-users] Request: etchdebug.log to configurable destination

[Pat O'B. <obr...@gm...>]

Well, I worked out a fix for this, although I am unsure if it's a good fix
or not.

First, here is the diff:

------------------------------>%------------------------------
wordup:server pobrien$ svn diff
Index: lib/etch/server.rb
===================================================================
--- lib/etch/server.rb  (revision 298)
+++ lib/etch/server.rb  (working copy)
@@ -203,12 +203,24 @@
     @facts = facts
     @tag = tag
     @debug = debug
-    @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log',
'etchdebug.log'))
-    if debug
-      @dlogger.level = Logger::DEBUG
-    else
-      @dlogger.level = Logger::INFO
+    @log_dir = nil
+    config_file = File.join(Etch::Server.configbase, 'etchserver.conf')
+    if File.exist?(config_file)
+      IO.foreach(config_file) do |line|
+        # Skip blank lines and comments
+         next if line =~ /^\s*$/
+         next if line =~ /^\s*#/
+        line.chomp
+        if line =~ /^log_dir:/
+          @log_dir = line.split.pop
+        end
+      end
     end
+
+    @log_dir ? @dlogger = Logger.new(File.join(@log_dir, 'etchdebug.log'))
:  @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log',
'etchdebug.log'))
+
+    debug ? @dlogger.level = Logger::DEBUG : @dlogger.level = Logger::INFO
+
     @fqdn = @facts['fqdn']

     if !@fqdn
------------------------------>%------------------------------

Here are my problems with the change: the first is that I'm not sure if this
config should go into <etchbase>/etchserver.conf or if it should be within
the RAILS_ROOT/config/ directory. This feels like something that should be
configured along with the rails app rather than in /etc/etchserver, but from
what I understand (and I am in no way, shape or form versed in rails) in
rails 2.x.x there is no generic config file to handle something like this.
There is a railscast (
http://railscasts.com/episodes/85-yaml-configuration-file) that touches upon
this, and they basically recommend RAILS_ROOT/config/config.yml along with
some other necessary changes to actually have the config file loaded.

Second, it looks like the <etchbase>/etchserver.conf file is currently only
used to define two things:  @@auth_enabled and @@auth_deny_new_clients =
nil, and the way these are loaded is by parsing the config file the same way
I parse it for the log_dir setting. I would prefer etchserver.conf file to
be a YAML file and we could load it that way, but I haven't looked too
deeply as to what else this file is used for so I don't really want to make
any recommendations on how to change it.

Third, I haven't looked hard enough to see whether or not anything else
references a log directory of any kind and because of my two issues above I
won't bother until there's a bit more discussion on it.

Other than that the change works fine and behaves as you would expect it to,
your etchserver.conf file would look like this:

log_dir: /whatever/directory/your/heart/desires

-pat




On Sun, Aug 7, 2011 at 10:19 PM, Pat O'Brien <obr...@gm...>wrote:

> I've never bothered to try this so I don't know if it's configurable or
> not. If it's not open a bug ticket and I'll take a look at it when I get
> some spare time this week.
>
> Just as a side note: my personal preference is to not have system logs end
> up in /var/log, but to each their own :)
>
> -pat
>
>
>
> On Wed, Aug 3, 2011 at 3:27 PM, Johnston, Nathaniel <
> Nat...@ca...> wrote:
>
>> Etch Mensches,
>>
>> I am not sure how best to do this, but I am trying to move my Etch logs
>> over to /var along with the other logfiles I have.  I noticed that on line
>> 205 of lib/etchserver.rb the etchdebug.log file is specified as:
>>
>> @dlogger = Logger.new(File.join(Rails.configuration.root_path, 'log',
>> 'etchdebug.log'))
>>
>> Is there a way that this could leverage the value of
>> config.log_path/paths.log or another configurable parameter so that we could
>> override the location of this file?
>>
>> --N.
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
>> The must-attend event for mobile developers. Connect with experts.
>> Get tools for creating Super Apps. See the latest technologies.
>> Sessions, hands-on labs, demos & much more. Register early & save!
>> http://p.sf.net/sfu/rim-blackberry-1
>> _______________________________________________
>> etch-users mailing list
>> etc...@li...
>> https://lists.sourceforge.net/lists/listinfo/etch-users
>>
>
>