Tutorial Four

 

Contents

 

 

1.       Introduction

The term scalability is very often misused. We define scalability as the following three aspects:

a.                   Scalability is the ability to fully utilize available processing power on a multiprocessor system (2, 4, 8, 32, or more processors).

b.                  Scalability is the ability to serve a large number of clients with sufficient performance.

c.                   Scalability is the ability to support a large variety of networks (phone line, cable/dsl, wireless modems as well as LAN) and machines (desktop pc, pocket pc and devices etc).

 

If you have already studied the previous three tutorials, you should understand that SocketPro has very good qualities in the above aspects b and c. In this tutorial, we are going to show you how good SocketPro is in the above aspect a. This tutorial is designed for you to take advantage of ATL COM object USocketPool.

 

2.       Fundamentals about COM object USocketPool
         

Similar to a thread pool in many aspects, a socket pool contains a set of sockets concurrently running within a few of threads. All of sockets can be shared and reused by thousands of clients. Each of threads hosts a few of sockets. All of sockets, even within the same thread, run with very good concurrency. The following figure shows a socket pool having three threads, and each of them hosts two USocket objects.

 

           

 

There is one key point with the socket pool object. When a service handler is attached to a client socket, all of requests with the handler are sent from one thread other than one thread in the pool, but their return results are always processed in one of the pool threads, because all of USocket events happen within the pool hosting threads. Under most of cases, sending a request is a fast step, but processing its returning result is usually slow. With help of socket pool, all of returning results are always processed with pooled threads concurrently. This improves application scalability.

 

To maximize performance and scalability, we must host a socket pool within a COM MTA apartment (free threaded) to avoid data marshalling across threads if possible. It is the reason that socket pool requires a MTA apartment. Additionally, a socket pool can contain 64 sockets at most, which is always far enough for all of applications.

           

It is noted that the code of the client application is ignored here because it has nothing special. You can understand it easily if you have already completed the tutorial one. This tutorial is focused on the server side development utilizing USocketPool object.

 

3.       Set up RemoteConnector.exe

           

To experiment this tutorial, first you need to set up RemoteConnector.exe, an attached window service written from C++. Please follow the short article at this site to make sure that the window service is running at the port 17001 and the library uodbsvr.dll is plugged into the application. We assume that you have either ACCESS or SQL server northwind database available to the RemoteConnector.exe.

 

4.       Basic steps to take advantages of USocketPool


To use USocketPool, follow the below steps:

First of all, create an instance of USocketPool and start the pool as below.

 

m_SocketPool = new USocketPoolClass();
m_SocketPool.StartPool(bThreadCount, bSocketsPerThread);

 

Next, build socket connections to a remote SocketPro server for an initial service.

 

m_SocketPool.ConnectAll("localhost", 17001, (int)USOCKETLib.tagServiceID.sidOleDB, "SocketPro", "PassOne", 0, false);

 

Next, bind each of USocket objects to one service handler or multiple service handlers having different service ids.

 

USocketClass ClientSocket = (USocketClass)m_SocketPool.LockASocket(0, null);

DBHandler.m_SocketPool = m_SocketPool;
CDBHandler DBHandler = new CDBHandler();
DBHandler.AttachSocket(ClientSocket);
m_SocketPool.UnlockASocket(ClientSocket);

 

Once the above steps are completed, the socket pool is ready to use. Here are steps to use the socket pool.

 

To send requests to a remote server, you’d lock a USocket object first as below. We are going to talk more about locking a USocket object with either main thread or worker thread in the coming Section.

           

//once a socket is locked, others can’t lock it or its bound db handler until it is unlocked

ClientSocket = (USocketClass)PoolSvs.m_SocketPool.LockASocket(0, null);

                             

Afterwards, find a bound handler through the locked USocket if successful.

 

CDBHandler DBHandler = (CDBHandler)PoolSvs.SeekDBHandler(ClientSocket);

 

Afterwards, batch all of requests and send them to a remote server.

 

ClientSocket.StartBatching();

DBHandler.m_Command.ExecuteSQL(m_strSQL, (short)UDBLib.tagCreatedObject.coRowset, (short)UDBLib.tagCursorType.ctStatic, 0);

DBHandler.m_Rowset.Open(null, 0, 0, 0, false);

DBHandler.m_Rowset.AsynFetch(true, 0, -1);

DBHandler.m_Rowset.Close();

 

//When the request is returned, we'll unlock a client socket and its bound handler

ClientSocket.DoEcho();

ClientSocket.CommitBatching(true);

 

Finally, once all of requests are processed, send results back to a client and unlock the locked USocket as shown in the below code.

 

case (short)USOCKETLib.tagBaseRequestID.idDoEcho:

{

nRtn = m_Command.Rtn;

if (nRtn >= 0)

                                nRtn = 0;

m_UQueue.Push(nRtn);

                if (nRtn < 0)

{

                               //attach DB error message

               m_UQueue.Save(m_Command.ErrorMsg);

}

                if (m_ClientSocket.Rtn != 0)

                {

               //send error message about socket connection instead

               m_UQueue.SetSize(0);

               m_UQueue.Push(m_ClientSocket.Rtn);

               m_UQueue.Save(m_ClientSocket.ErrorMsg);

}

               

nRtn = PoolPeer.SendReturnData(m_nCurrentRequestID, m_UQueue);

 

//make sure a locked client socket is unlocked

                m_SocketPool.UnlockASocket(m_ClientSocket);

}

 

As described in the above and previous tutorials, your return results are processed inside USocket event OnRequestProcessed. Here is a sample code snippet.

 

case (short)UDBLib.tagDBRequestID.idRowsetOpen:

{

                m_UQueue.Push(m_Rowset.Rtn);

                if(m_Rowset.Rtn < 0)

                {

                                m_UQueue.Save(m_Rowset.ErrorMsg);

                }

                else

                {

                                int nCol;

                                int nCols = m_Rowset.GetCols();

                                for(nCol = 1; nCol<=nCols; nCol++) //One-based index for column

                                {

                                                m_UQueue.Push((object)m_Rowset.GetColName(nCol));

                                }

                }

                PoolPeer.SendReturnData(CPoolSvs.idColNames, m_UQueue.GetBuffer(), m_UQueue.GetSize());

}

 

 

5.         Lock a USocket object within either main or worker thread

 

Beginning with SocketPro version 4.8.1.1, you can lock a USocket object within either main or worker thread. To maximize SocketPro server performance and scalability with this sample tutorial, you can try to lock a USocket object first within main thread inside the virtual function CClientPeer::OnDispatchingSlowRequest as below.

 

protected override void OnDispatchingSlowRequest(short sRequestID)

{

USocketClass ClientSocket;

m_UQueue.Load(ref m_strSQL);

 

//This virtual function is called within main thread

try

                {

//Timeout should be zero. Otherwise, there is deadlock potentially

//when there is no socket available in socket pool for locking.

//Make sure that main thread is never blocked in SocketPro!

                                ClientSocket = (USocketClass)m_SocketPool.LockASocket(0, null);

                }

catch (COMException eCOM)

{

                               //if no socket is locked, OnSlowRequestArrive will be called

                                return;

}

//send requests from main thread directly

SendRequests(sRequestID, ClientSocket);

 

//There is no need to dispatch current slow request to a worker thread any more

DropCurrentSlowRequest();

 

//This new feature (SocketPro version 4.8.1.1) reduces thread context switches!

}

           

The above code sees if a USocket object is available from a socket pool. If a USocket object is available to use, we send requests directly from main thread without dispatching the current slow request to a worker thread. This new feature actually speeds up sending requests because the code above saves dispatching a request onto a worker thread and extra thread context switches. If no USocket object is available to use, the main thread will dispatch a slow request onto a worker thread for processing within the virtual function OnSlowRequestArrive.

 

Inside the function OnSlowRequestArrive, we need to monitor socket closing event and special client request Cancel. Once finding the closing event or request Cancel, we should stop locking a USocket object and exit the function as below.

 

protected override int OnSlowRequestArrive(short sRequestID, int nLen)

{

USocketClass ClientSocket;

                do

                {

                                try

                                {

                                                ClientSocket = (USocketClass)m_SocketPool.LockASocket(50, null);

                                                break;

                                }

                                catch (COMException eCOM)

                                {

                                                if (IsCanceled || IsClosing())

                                                {

                                                                //If we find a client socket is going to shut down or current request is canceled,

                                                                //stop locking a usocket object.

                                                                return 0;

                                                }

                                }

                } while (true);

                SendRequests(sRequestID, ClientSocket);

return 0;

}

           

The most important point is that we shouldn’t lock a USocket object within main thread with a long timeout, which may lead to a potential deadlock, because SocketPro uses main thread to drive many events and you should never block it with a lengthy action by design.

 

6.         CSocketPool<THandler>

 

This tutorial uses ATL COM UDB.dll, but coding logical is exactly the same with service handler created from the class CRequestAsynHandlerBase. To simplify use of socket pool object, SocketProAdapter has a generic class for .NET development or a template for C++ development.

 

Take the asynchronous handler in the tutorial two. You can use the following code.

 

//public class CTOne : CRequestAsynHandlerBase ......

CSocketPool<CTOne> poolTOne;

poolTOne = new CSocketPool<CTOne>();

if (poolTOne.StartSocketPool("localhost", 20901, "SocketPro", "PassOne", 3, 0, tagEncryptionMethod.MSTLSv1, false))

{

//lock a client socket

                CTOne Tone = poolTOne.Lock();

                if (Tone != null)

                {

                                //send all of requests one-by-one or by batch here

                                object strEcho = Tone.Echo("This is test");

                                int nData = Tone.QueryCount();

                                nData = Tone.QueryGlobalCount();

 

                                int nData2 = 0;

                                int nData3 = 0;

                                Tone.GetAllCounts(out nData, out nData2, out nData3);

                                 poolTOne.Unlock(Tone); //unlock the socket

                }

 

                //do something else .....

 

                poolTOne.ShutdownPool(); //don't need the pool any more

}

 

7.       Experiment the ability to utilize available processing power on a multiprocessor system

           

To experiment the ability to available processing power on a multiprocessor system with SocketPro, you don’t have to use a multiprocessor system. You can simply test it over a normal machine. If your machine has a processor enabled with Intel Hyper-Threading technology, the machine is good enough. Suppose that you have such a machine running on MS XP platform. We’ll have three tests.

 

Start the server application SocketPool.exe, and play it for a while so that you understand what it does. Now, let’s experiment it with two instances of SPoolClient.exes. All of the applications can be running in the same machine.

 

Test One – Single thread and single socket -- First, set both text boxes Thread Count and Sockets per Thread to one and click the button Start Server in the application SocketPool.exe. Next, start two instances of SPoolClient.exe, click the button Connect to build a socket connection to SocketPool.exe and check Rowset for the two instances of SPoolClient.exe. Input one SQL statement Select * from Products, Customers for one SPoolClient.exe and Select * from Products for the other. You can click the button Execute SQL for the first SPoolClient.exe first, and quickly for the second. You will see that what happens is that two SQL statements are queued and executed in the server application SocketPool.exe. My performance monitor shows 50% CPU usage only. For this case, multiple requests of different clients are always queued.

 

Test Two – Single thread and two sockets -- This test is very similar to the above one, except that you should set the text box Sockets per Thread to two for the server application SocketPool.exe. Once playing it, you will find that the second SQL statement can be quickly completed because the first one takes much longer time to process. You will see the two SQL statements processed in parallel even though we started the pool with one thread only. SocketPro is very different from other common frameworks. It is impossible or impractical to do this kind of work with other common frameworks. SocketPro is written from 100% non-blocking socket. On thread can host multiple non-blocking sockets and all of them could run with an excellent concurrency. If you change the second SQL statement into the first one and run the two SQL statements through the two instances of SPoolClient.exe at the same time, you will find the two can be also completed at the about same time. Further, you may see that the CPU usage is over 80% instead of 50% and two processors are well utilized at the same time.

 

Test three – Multiple threads and One Socket per Thread -- To better experiment this, you need to run one instance of SocketPro.exe on an independent machine, and multiple instances of SPoolClient.exe applications over different client machines. All of them are networked with local area network having bandwidth 100 mbps. Start SocketPool.exe with multiple threads but one socket per thread (you can also do multiple sockets per thread). All of SPoolClient.exe are executed with the first SQL statement Select * from Products, Customers. Once executing the SQL statement from all of client machines at a time, you will see all of SQL statements are processed concurrently and CPU usage is nearly 100%. If you can’t see the 100% CPU usage because of performance of SocketPro, you can see it after checking Zip on SPoolClient.exe and playing it again. You also see how fast SocketPool can process.

 

By completing the above three experiments, you can see what scalability SocketPro can achieve. SocketPro can fully utilize all of processors on a server machine.

 

8.         Advantages of USocketPool -- Scalability

 

a.                   A limited number of precious internal socket objects can be reused with all of thousands outside connected clients forever. Also, all of internal service handlers are created one time only, and are reused forever by all of outside connected clients. The consumed resources are greatly reduced because all of resources can be shared over all of outside clients.

b.                  The associated backend database sessions are limited too, which causes not only low resources required but also paying much less money to database management vendors because the cost fee is dependent on the number of database sessions.

c.                   It simplifies my development too. You don’t have to create a thread pool. The COM USocketPool object does all of hard works for you. It also reduces data synchronization complexity in absolutely most cases.

d.                  All of socket sessions can run with excellent concurrency. If my server has multiple processors, all of processors can be taken advantage of.

 

9.       The cases for USocketPool

 

a.         When you develop a distributed application system with four or more tiers (front GUI application tier, front business tier written from SocketPro server like this tutorial project, backend business tier like RemoteConnector.exe and database tier), it is very beneficial to use USocketPool at the second tier assuming the very front application is the first tier.

b.         When you develop an asp.net application with three tiers (front asp.net application tier, business tier and database tier), you should use USocketPool, which will dramatically improve your web performance and scalability.

c.         When you develop a front window application that requires listing a lot of items with a tree or list view control and that also runs in a processor enabled with Intel Hyper-Threading technology, you can use USocketPool to accelerate GUI performance. At this time, many GUI applications can’t fully take advantage of Intel Hyper-Threading technology and one of hyper processor is wasted. If you use USocketPool, you can easily process returned results in a hyper processor and list a lot of items with the other hyper processor so that two hyper processors can be fully utilized.

 

Overall, you’d better use USocketPool if your project belongs to the first two cases (a and b). For the case c, the benefits of USocketPool are not obvious if your window application doesn’t list a lot of items.

 

10.     FAQs

 

a.         Can I create multiple socket pools within one application because I want to pool all of my services like this tutorial and each of them uses its own socket pool?

                       

Absolutely yes!

 

b.         Can I use one pool of sockets to handle all of different services?

Yes, you can. However, you may need to check current service id and call IUSocket::SwicthTo to ask for a service if current service id is not equal to my service id as shown in the tutorial 3.