SD-WAN

Hi, I want to develop a module for HPVAN 2.7.0 that uses threads to collect counters from switches. Under my model, a flow is constituted by a Match and a list of DPIDs. What I want to do is to let 'assign' a thread to each DPID such that, each thread will be in charge of constantly sending/receiving OF request/reply messages to/from the assigned DPID.

Here's an initial code snippet:

private List<Flow.CounterElem> getFlowCounters(long cookie, long cookieMask, Flow v) throws OpenflowException, InterruptedException {

	List<Flow.CounterElem> counters = Collections.synchronizedList(new ArrayList<Flow.CounterElem>());
	List<DataPathId> randomDpids = new ArrayList<DataPathId>(v.dpidOnlyList());  //copy array, not reference to original array
	Collections.shuffle(randomDpids, new Random(System.nanoTime()));
	for (DataPathId dp: randomDpids){
		StatsCollectorByCookie thread = new StatsCollectorByCookie(counters, dp, cookie, cookieMask);
		thread.run();
         }	
         return counters;
}

public class StatsCollectorByCookie implements Runnable {
	private List<Flow.CounterElem> counterlist;
	private DataPathId dpid;
	private long cookie;
	private long cookieMask;
		
	public StatsCollectorByCookie(List<Flow.CounterElem> result, DataPathId dp, long cookie, long mask){
		this.counterlist = result;
		this.dpid = dp;
		this.cookie = cookie;
		this.cookieMask = mask;
	}
		
	@Override
	public void run() {
    	Flow.CounterElem elem;
    	try {
    		elem = getDatapathCounters(cookie, cookieMask, dpid);  //Create MBodyMutableFlowStatsRequest, sends it to switch and return response data
    		if (elem != null)
    			counterlist.add(elem);
    	} catch (OpenflowException | InterruptedException e) {
    		log.info("There's an error with thread collecting data from ", this.dpid.toString());
    		e.printStackTrace();}
	}		
}

 I'd like to have some feedback with respect to the following points:

1. Is this the right way to spawn threads in HP VAN? The programming guide does mention something about threads but does not provide any code on how to properly implement them. Also, there are a couple of Thread - Executor classes in HP's API, so I am wondering if I should use them instead.

2. As a follow-up from 1. Should I rather use ThreadPool/ExecutorService?

3. Assuming those counters will be queried constantly and an external service will get them via REST, is the above approach an efficient way of creating threads? or I am better off spawing a thread (T) whenever a flow is created (i.e. I will have a thread per flow rather than a thread per dpid in flow) and let T handle OF messages for that flow and put that information in a variable which can be further accessed via REST.

Thanks!

Hello checho,

I'm not certain that the code you posted will actually spawn another thread. I think that what your code is doing is using the Java framework (Runnable) without getting the Runnable to run in a separate thread.

To accomplish what you've described, I'd recommend the following:

1. Create an OSGI service that provides the ability to store and look up the data (similar to NodeService or LinkService).

2. Create a REST API Resource file that uses a reference to your OSGI service to provide the data externally.

3. When your service is initialized in the @Activate method, spawn a thread that will monitor each datapath or each flow.

Typically the number of flows is much higher than the number of datapaths, so you'd want to spawn a thread per datapath. If you spawn too many threads, it may consume too many system resources (memory/CPU) to handle them all.

You'll want to spawn a single thread like this:

StatService statService = ...; // To be created for collecting/retrieving stats
BlockingQueue<Runnable> threadQueue = new LinkedBlockingQueue<>(MAX_QUEUED_THREADS);

ThreadPoolExecutor exec = new ThreadPoolExecutor(MAX_THREADS, MAX_THREADS, 1, TimeUnit.MINUTES, threadQueue, new ThreadPoolExecutor.DiscardPolicy());
exec.setThreadFactory(new NamedThreadFactory("checho"));

for (int i = 0; i < num_threads; i++) {
    exec.submit(new StatsCollectorByCookie(statService, dpId));
}

Then each StatsCollectorByCookie could constantly monitor the datapath, then add collected stats to the StatService.

If you want StatsCollectorByCookie() to run periodically and only collect stats once each time it runs, you could use java.util.Timer.scheduleAtFixedRate() and make your StatsCollectorByCookie a TimerTask.

Be sure that your code correctly handles a datapath disconnect/reconnect, in case the switch reboots or has a control-plane disruption.

Shaun thanks for your reply. 

I do have steps 1 and 2 implemented. With respect to 3 I'd like to make sure I understand the code you posted. 

1. There is an independent blocking queue per datapath.
2. My original thought was that I would have one thread per datapath but instead, according to your code, what I create per datapath is a ThreadPoolExecutor and thus, multiple threads per datapath. Am I correct?

Update

Some more thoughts, I do need to periodically collect the statistics per datapath. In addition, I need to constantly update a data structure (currently a ConcurrentHashMap) with the latest counters (values) per datapath (keys) that match a cookie/cookiemask. Ideally, a thread (or a threadpool) should update the value of one and only one key but I am not sure I can enforce that accurately/efficiently. 

In addition, I read about TimerTasks but also found the ScheduledThreadPool (it implements ScheduledExecutorService) which is part of Java's concurrent library and I am leaning towards the latter, is there any reason to prefer TimerTasks?

Hi checho,

According to the ScheduledThreadPoolExecutor javadoc, it sounds like that's what you'll want:

This class is preferable to Timer when multiple worker threads are needed, or when the additional flexibility or capabilities of ThreadPoolExecutor (which this class extends) are required.

In my example code above, I was just giving a framework for how you'd code multiple threads to be spawned. In more detail, I think a good plan would be the following, in addition to what's written above:

• During initialization of StatService, call ControllerService.getAllDataPathInfo(). Create a StatsCollectorByCookie for each datapath, then store a reference to the StatsCollectorByCookie in a structure. Run each StatsCollectorByCookie using a ScheduledThreadPoolExecutor.
• Create a DatapathListener that listens for OpenflowEventType.DATAPATH_READY, then spawn a new StatsCollectorByCookie (if one hasn't already been recorded in the structure above).

Shaun

Thank you Shaun. I think I am on the right track then.