Setting up MATLAB clients

Congratulations! You've received an exclusive limited-time opportunity to be one of the first people to use NeuroCluster 2.0 beta alpha zeta.

Setup

Prerequisites

  • MATLAB R2009b w/ the GT license and the Parallel Computing Toolbox installed
  • You are hardwired (i.e. via a ethernet cable) to the Neuro network (this is not strictly a requirement - you can do it from another network, but it's a bit harder - ask Dustin about it)
  • You have patience

Configuration of MATLAB

  1. Start your MATLAB client GUI
  2. Run the following code (you may want to put this in your startup.m file):
    disp('Running startup.m');
    MyExternalIP=urlread('http://ip.dustin.li/');
    pctconfig('hostname',MyExternalIP);
    disp('Startup.m complete.');
  3. From the Parallel menu choose Manage Configurations...
  4. Now that you're in the Configurations Manager, choose File->New->jobmanager
  5. Configure the Job manager hostname (brain.neuro.gatech.edu) and Job manager name (NeuroCluster) as shown below (use brain.neuro.gatech.edu instead of vm1.neuro.gatech.edu). Click OK.
  6. Choose NeuroCluster as the default configuration in the Configurations Manager.


Testing the configuration

Simple playback

You're almost there. Click the "Start Validation" button in the lower right hand corner of the configurations manager. If everything is setup correctly you should see 4 green checkmarks appear. 

On the command line, run:
matlabpool 4
spmd
pause(labindex);
disp(labindex);
end
matlabpool close
 
You should see 4 "labs" sequentially respond with their indices.

Benchmarking speedup

Now let's see how the cluster performance scales with additional workers for an embarrassingly parallel job. This code uses pctdemo_task_blackjack. Run dbtype pctdemo_task_blackjack to see what's happening behind the curtain.

Code:
pctdemo_aux_parforbench.m:
function S = pctdemo_aux_parforbench(numHands, numPlayers, n)
%PCTDEMO_AUX_PARFORBENCH Use parfor to play blackjack.
%   S = pctdemo_aux_parforbench(numHands, numPlayers, n) plays
%   numHands hands of blackjack numPlayers times, and uses no
%   more than n MATLAB(R) workers for the computations.

%   Copyright 2007-2009 The MathWorks, Inc.
S = zeros(numHands, numPlayers);
parfor (i = 1:numPlayers, n)
    S(:, i) = pctdemo_task_blackjack(numHands, 1);
end

Script:
%http://www.mathworks.com/products/parallel-computing/demos.html

%http://www.mathworks.com/products/parallel-computing/demos.html?file=/prod
%ucts/demos/shipping/distcomp/paralleldemo_parfor_bench.html

% modified by Dustin Li, 04/20/2010

poolSize = matlabpool('size');
if poolSize == 0
    error('distcomp:demo:poolClosed', ...
        'This demo needs an open MATLAB pool to run.');
end

% parallelize this task
numHands = 500;
numPlayers = 6;
fprintf('Simulating each player playing %d hands of blackjack.\n', numHands);
t1 = zeros(1, poolSize);
for n = 1:poolSize
    tic;
        pctdemo_aux_parforbench(numHands, n*numPlayers, n);
    t1(n) = toc;
    fprintf('%d workers simulated %d players in %3.2f seconds.\n', ...
            n, n*numPlayers, t1(n));
end

% make sure we aren't be limited by communication overhead
tic;
pctdemo_aux_parforbench(numHands, 10*numPlayers, 1);
clusterToc = toc;
fprintf('1 cluster workers simulated %d players in %3.2f seconds.\n', ...
    10*numPlayers, clusterToc);

% sequential loop
% tic;
%     parfor z = 0
%         S = zeros(numHands, numPlayers);
%         for i = 1:numPlayers
%             S(:, i) = pctdemo_task_blackjack(numHands, 1);
%         end
%     end
% t1(1) = toc;
% fprintf('Ran in %3.2f seconds using a sequential for-loop.\n', t1(1));

tic;
S = zeros(numHands, 60);
for i = 1:60
    S(:, i) = pctdemo_task_blackjack(numHands, 1);
end
localToc = toc;
fprintf('1 local worker simulated 60 players in %3.2f seconds.\n', localToc);
disp('(Old cluster: 1 worker, 60 players, 11.89 seconds.)');

speedup = (1:poolSize).*t1(1)./t1;
fig = pctdemo_setup_blackjack(1.0);
set(fig, 'Visible', 'on');
ax = axes('parent', fig);
x = plot(ax, 1:poolSize, 1:poolSize, '--', ...
    1:poolSize, speedup, 's', 'MarkerFaceColor', 'b');
t = get(ax, 'XTick');
t(t ~= round(t)) = []; % Remove all non-integer x-axis ticks.
set(ax, 'XTick', t);
legend(x, 'Linear Speedup', 'Measured Speedup', 'Location', 'NorthWest');
xlabel(ax, 'Number of MATLAB workers participating in computations');
ylabel(ax, 'Speedup');

%%
% now plot single thread results
figure(2);
bar([7.37 localToc 11.89]);
set(gca,'XTickLabel',{'New cluster','Local','Old cluster'});
ylabel('Execution time (s)');
title('Single threaded performance');
set(gcf, 'Name', 'Single threaded performance');


Example output for a matlabpool of 10:
Simulating each player playing 500 hands of blackjack.
1 workers simulated 6 players in 0.93 seconds.
2 workers simulated 12 players in 0.93 seconds.
3 workers simulated 18 players in 0.95 seconds.
4 workers simulated 24 players in 1.35 seconds.
5 workers simulated 30 players in 1.06 seconds.
6 workers simulated 36 players in 0.97 seconds.
7 workers simulated 42 players in 1.01 seconds.
8 workers simulated 48 players in 1.03 seconds.
9 workers simulated 54 players in 1.05 seconds.
10 workers simulated 60 players in 1.78 seconds.
1 cluster workers simulated 60 players in 8.77 seconds.
1 local worker simulated 60 players in 10.86 seconds.
(Old cluster: 1 worker, 60 players, 11.89 seconds.)

Speedup is fairly linear in this case. 

The performance of the new cluster is much improved over the old cluster.

Performance

Compared to the old cluster, sequential performance should also be much higher. The new cluster runs 64-bit Ubuntu Linux on MATLAB r2009b, instead of Mac OS X 10.4 on MATLAB r2008a.

Numbers

Old:
>> for n=1
tic;
pctdemo_task_blackjack(5000,1);
toc
end
Elapsed time is 1.984382 seconds.

New:
>> for n=1
tic;
pctdemo_task_blackjack(5000,1);
toc
end
Elapsed time is 1.228749 seconds.

This particular code yields a 38% speedup!

Caveats

  • At this time, the NeuroCluster is quite finicky with connections outside of the Neuro network (i.e. it's usually not as simple as VPNing into the Neuro net). For external connections, it may be necessary to run this script as a startup file in MATLAB:
% Dustin's startup.m
% 02/2010
disp('Running startup.m');
MyExternalIP=urlread('http://ip.dustin.li/');
pctconfig('hostname',MyExternalIP);
disp('Startup.m complete.');
  • There also seems to be a problem if you're behind certain NATs, even if DMZ is set up appropriately. I had a problem with a Netgear RP614 v3 wired to the Neuro network.
  • Occasionally, the cluster will get stuck, and not start new jobs even when there are idle nodes. findResource may show that there are finished jobs, pending jobs, but no running jobs. In this case, use admincenter to stop and resume each worker.

Further information

See the MATLAB documentation for further information. This is a good place to start:
doc distcomp
Adam Charles also has a great tutorial on parallelization:

Scheduling

We use a manual "scheduler" to coordinate time on the cluster. Throw your name on the Google Calendar if you want time. Click the "+" button in the lower right corner of the calendar below to add it to your calendars. Ask Adam, Jeff, or Dustin to add you so that you can make changes.

Google Calendar


ċ
NeuroClusterConfiguration.mat
(1k)
Dustin Li,
May 18, 2010, 10:43 AM
Comments