Slides for the talk tak at lpw-2011
-
The First
Thing Tak
Did ...
-
Tak?
-
Fab
-
fabfile.org
-
Mass SSH
-
Runs ...
commands
-
local()
remote()
-
BORING
-
Sysadmins
like it
-
Ok ...
-
mcollective
-
Message
queue backed
-
Runs ...
commands
-
BORING
-
(there's
also Rex
in perl)
-
What's the
general
concept?
-
Messaging
Jobs
-
Ok. Step
back.
-
"Run [this]
on [these]
systems"
-
Semi
independent
agents
-
Running
code on
remote
systems
-
But ...
-
Dependency
management
-
Dependency
management
for ad-hoc
code sucks
-
App::FatPacker
-
Have to
pack each
script ...
-
... and
copy it
to all
the hosts
-
BORING
-
perl
to the
rescue
-
__END__
-
"script is
done now"
-
Which
means ...
-
ssh host
perl -
-
cat script
echo __END__
-
Then
what?
-
The Second
Thing Tak
Did
-
Need a
protocol
-
Simple
-
Human
readable
-
Already
written
-
HTTP?
-
Ugh.
-
Line
based?
-
Ok but ...
-
Not writing
a parser
-
JSON!
-
[ "type", "payload", "here" ]
-
Reliability
-
Three
message
types
-
Message.
Request.
Response.
-
Wait.
-
What
about
progress?
-
Response
=
Progress
Result
-
Message
Request
Progress
Result
-
Why have
requests
at the
protocol
level?
-
Requests
must
always
complete
-
Tracking
-
Tracking
across
hosts
-
Tracking
over
time
-
Message
routing
-
Central
dictionary?
-
Why?
-
Opaque
routing
-
[ "request",
"service-name",
"type",
"payload",
...
]
-
Service
name?
-
Router
-
my ($self, $service_name, @msg) = @_;
$services{$service_name}->send(@msg);
-
Meta
service
-
[ "request",
"meta",
"register",
"service-name",
"Service::Class",
...
]
-
Client
-
# do or die()
$client->do(
service_name
=> request_type
=> @payload
);
-
# result object
$client->result_of(
service_name
=> request_type
=> @payload
);
-
# return data or throw
$result->get;
# get error if any
$result->exception;
-
my $req = $client->start({
on_result => sub { ... },
on_progress => sub { ... },
}, @payload);
-
# And finally
Tak->await_all(
@requests
);
-
-
A small
diversion
-
Event
loops
-
while (my $line = <$fh>) {
...
}
-
Parallel
-
Arse.
-
POE
IO::Async
AnyEvent
-
More than
I needed
-
Steal from
AnyEvent?
-
AnyEvent::Impl::Perl
-
select()
-
Handle
vectors
-
vec() ?
-
Nooo ...
-
# we parse the bitmask by first
# expanding it into a string of bits
for (unpack "b*", $vec[$_]) {
-
# and then repeatedly matching
# a regex against it
while (/1/g) {
-
# and use the resulting string position as fd
$_ && $_->[2]()
for @{ $fds->[W][(pos) - 1] || [] };
-
....
-
....
....
-
Fuck that,
I'll just use
IO::Select
-
-
tak
-
App::Tak
-
Tak::Script
-
Custom
commands
-
Tak::MyScript
-
Takfile
-
tak -h user@host command
-
sub each_command {
my ($self, $remote) = @_;
-
$remote?
-
ConnectorService
-
Net::OpenSSH
-
(and later
Net::SSH::Perl
for win32)
-
$ssh->open2('perl', '-');
-
$kid_in->print(
io($fatscript)->all,
"__END__\n"
);
-
$fatscript?
-
fatpack tree $(fatpack packlists-for
strictures.pm Moo.pm Algorithm/C3.pm
MRO/Compat.pm Class/C3.pm JSON/PP.pm
Log/Contextual.pm Data/Dumper/Concise.pm)
-
fatpack tree $(fatpack packlists-for
strictures.pm Moo.pm Algorithm/C3.pm
MRO/Compat.pm Class/C3.pm JSON/PP.pm
Log/Contextual.pm Data/Dumper/Concise.pm)
fatpack file
-
fatpack tree $(fatpack packlists-for
strictures.pm Moo.pm Algorithm/C3.pm
MRO/Compat.pm Class/C3.pm JSON/PP.pm
Log/Contextual.pm Data/Dumper/Concise.pm)
fatpack file
echo "use Tak::STDIOSetup;
Tak::STDIOSetup->run;"
-
$remote->do(
meta => ensure
=> eval_service
=> 'Tak::EvalService'
);
-
Wait.
-
We didn't
fatpack
that.
-
Far side
has a
loopback
router
-
$local->ensure(
module_sender
=> 'Tak::ModuleSender'
);
-
$remote->ensure(
module_loader => 'Tak::ModuleLoader',
expose => {
module_sender
=> [ 'remote', 'module_sender' ]
}
);
-
expose
passes
service
references
-
no global
router
required
-
Tak::ModuleSender
-
sub handle_source_for {
my ($self, $module) = @_;
my $io = (find module);
return scalar $io->all;
}
-
Tak::ModuleLoader
-
sub {
my $result = (ask sender);
my $code = $result->get;
open my $fh, '<', \$code;
return $fh;
}
-
push @INC, $hook;
-
sub each_exec {
my ($self, $remote, @cmd) = @_;
$remote->ensure(
command_service
=> 'Tak::CommandService'
);
-
my $result = $remote->do(
command_service
=> exec
=> \@cmd
);
-
sub handle_exec {
my ($self, $command) = @_;
my $code;
my ($stdout, $stderr) = capture {
$code = runx(EXIT_ANY, @$command);
};
-
Sequential
-
:(
-
sub every_exec {
my ($self, $remotes, @command) = @_;
my @requests;
foreach my $remote (@$remotes) {
push @requests, $remote->start(
...
);
}
Tak->await_all(@requests);
}
-
Streaming?
-
on_progress
=> sub {
-
Optional
streaming
-
sub every_exec (stream|s) {
my ($self, $remotes, $options, @command) = @_;
-
Getopt::Long
spec in
prototype!
-
sub every_exec (stream|s) {
my ($self, $remotes, $options, @command) = @_;
...
if ($options->{stream}) {
-
tak -h user@host
exec --stream tail -f ...
-
tak -h user@host
exec -s tail -f ...
-
Running
perl code
remotely?
-
Tak::ObjectService
-
my $oc = Tak::ObjectClient->new(
remote => $remote
);
my $obj = $oc->new_object(
$class => \%args_to_new
);
-
$json->convert_blessed;
-
$json->convert_blessed;
local *UNIVERSAL::TO_JSON = sub {
-
$json->decode(
$json->encode($msg)
);
-
$json->filter_json_single_key_object(
__proxied_object__ => sub { ... }
);
-
Local proxies
send method
call requests
-
sub each_get_homedir {
my ($self, $remote) = @_;
my $oc = Tak::ObjectClient->new(
remote => $remote
);
my $dot = $oc->new_object(
'Path::Class::Dir'
);
$home = $dot->absolute->stringify;
-
Experimentation
-
Run, swear,
tweak, run,
swear ...
-
tinyrepl
-
Eval::WithLexicals
-
Pure perl
-
Fatpackable
-
Tak::EvalService
-
eval {
($stdout, $stderr) = capture {
@ret = $eval->eval($perl);
};
};
-
Tak::REPL->new(
remote => $remote
)->run
-
$ tak -h user@host repl
re.pl$
-
Tak went
to CPAN
... finally
-
#perl-tak
exists on
freenode
-
Try it.
Break it.
Tell me
about it.
-
The Third
Thing Tak
Did
-
__END__
-
Thank You
IRC:mst
mst@shadowcat.co.uk
@shadowcat_mst