Skip to content

Cross-platform C++ framework for scalable, asynchronous, distributed client-server applications.

License

Notifications You must be signed in to change notification settings

cuongtv51/solidframe

 
 

Repository files navigation

SolidFrame

Cross-platform C++ framework for scalable, asynchronous, distributed client-server applications.

License

Boost Software License - Version 1.0 - August 17th, 2003

Resources

Prerequisites

Supported platforms

  • Linux - (tested on latest Fedora i686/x86_64 and Raspian on Raspberry Pi 2 armv7l) - gcc
  • FreeBSD - (tested on FreeBSD/PcBSD 10.3) - llvm
  • Darwin/OSX - (waiting for XCode 8 with support for thread_local) - llvm
  • Windows - (partial) - latest VisualStudio

Libraries

  • solid_system:
    • Wrappers for socket/file devices, socket address, directory access
    • Debug log engine
  • solid_utility:
    • Any - similar to boost::any
    • Event - Event class containing an ID a solid::Any object and a Category (similar to std::error_category)
    • InnerList - bidirectional list mapped over a vector/deque
    • Stack - alternative to std::stack
    • Queue - alternative to std:queue
    • WorkPool - generic thread pool
  • solid_serialization: binary serialization/marshalling
    • binary::Serializer
    • binary::Deserializer
    • TypeIdMap
  • solid_frame:
    • Object - reactive object
    • Manager - store services and notifies objects within services
    • Service - store and notifies objects
    • Reactor - active store of objects - allows objects to asynchronously react on events
    • Scheduler - a thread pool of reactors
    • Timer - allows objects to schedule time based events
    • shared::Store - generic store of shared objects that need either multiple read or single write access
  • solid_frame_aio: asynchronous communication library using epoll on Linux and kqueue on FreeBSD/macOS
    • Object - reactive object with support for Asynchronous IO
    • Reactor - reactor with support for Asynchronous IO
    • Listener - asynchronous TCP listener/server socket
    • Stream - asynchronous TCP socket
    • Datagram - asynchronous UDP socket
    • Timer - allows objects to schedule time based events
  • solid_frame_aio_openssl: SSL support via OpenSSL
  • solid_frame_file
    • file::Store - a shared store for files
  • solid_frame_ipc: asynchronous Secure/Plain TCP inter-process communication engine (IPC library)
    • ipc::Service - asynchronously sends and receives messages to/from multiple peers.

Installation

Following are the steps for:

  • fetching the SolidFrame code
  • building the prerequisites folder
  • building and installing SolidFrame
  • using SolidFrame in your projects

Please note that boost framework is only used for building SolidFrame. Normally, SolidFrame libraries would not depend on boost.

Linux/macOS/FreeBSD

System prerequisites:

  • C++11 enabled compiler: gcc-c++ on Linux and clang on FreeBSD and macOS (minimum: XCode 8/Clang 8).
  • CMake

Bash commands for installing SolidFrame:

$ mkdir ~/work
$ cd ~/work
$ git clone [email protected]:vipalade/solidframe.git
$ mkdir extern
$ cd extern
$ ../solidframe/prerequisites/prepare_extern.sh
# ... wait until the prerequisites are built
$ cd ../solidframe
$ ./configure -e ~/work/extern --prefix ~/work/extern
$ cd build/release
$ make install
# ... when finished, the header files will be located in ~/work/extern/include/solid
# and the libraries at ~/work/extern/lib/libsolid_*.a

If debug build is needed, the configure line will be:

$ ./configure -b debug -e ~/work/extern --prefix ~/work/extern
$ cd build/debug

For more information about ./configure script use:

$ ./configure --help

Windows

Windows is not yet supported.

Overview

SolidFrame is an experimental framework to be used for implementing cross-platform C++ network enabled applications or modules.

The consisting libraries only depend on C++ STL with two exceptions:

  • solid_frame_aio_openssl: which obviously depends on OpenSSL
  • solid_frame_ipc: which, by using solid_frame_aio_openssl implicitly depends on OpenSSL too.

In order to keep the framework as dependency-free as possible some components that can be found in other libraries/frameworks were re-implemented here too, most of them being locate in either solid_utility or solid_system libraries.

The next paragraphs will briefly present every library.

solid_system

The library consists of wrappers around system calls for:

The most notable component is the debug log engine which allows sending log records to different locations:

  • file (with support for rotation)
  • stderr/stdlog
  • socket endpoint

The debug engine defines some macros for easily specify log lines. The macros are only active (do something) when SolidFrame is compiled with SOLID_HAS_DEBUG (e.g. maintain and debug builds). Also, the debug engine has support for registering modules and for specifying enabled log levels. Here is an example:

	Debug::the().levelMask("view");
	Debug::the().moduleMask("frame_ipc:iew any:ew");
	Debug::the().initStdErr();
	//...
	//a log line for module "any"
	edbg("starting aio server scheduler: "<<err.message());
	//...
	//a log line for a predefined module "frame_ipc" aka Debug::ipc
	idbgx(Debug::ipc, this<<" enqueue message "<<_rpool_msg_id<<" to connection "<<this<<" retval = "<<success);

In the above code we:

  • Set the global levelMask to "view" (V = Verbose, I = Info, E = Error, W = Warning).
  • Enable logging for only two modules: "frame_ipc" and "any" (the generic module used by [v/i/e/w]dbg() macros). For "frame_ipc" restrict the level mask to {Info, Error, Warning} and for "any" restrict it to only {Error and Warning}.
  • Configure the debug log engine to send log records to stderr.
  • Send a log error line for "any" module.
  • Send a log info line for "frame_ipc" module.

The Debug engine allows for registering new modules like this:

	static const auto my_module_id = Debug::the().registerModule("my_module");
	//...
	//enable the module 
	Debug::the().moduleMask("frame_ipc:iew any:ew my_module:view");
	//...
	//log a INFO line for the module:
	idbgx(my_module_id, "error starting engine: "<<error.mesage());

or like this:

	static unsigned my_module_id(){
		static const auto id = Debug::the().registerModule("my_module");
		return id;
	}
	
	//...
	//enable the module 
	Debug::the().moduleMask("frame_ipc:iew any:ew my_module:view");
	//...
	//log a INFO line for the module:
	idbgx(my_module_id(), "error starting engine: "<<error.mesage());

solid_utility

The library consists of tools needed by upper level libraries:

  • any.hpp: A variation on boost::any / experimental std::any with storage for emplacement new so it is faster when the majority of sizeof objects that get stored in any<> fall under a given value.
  • event.hpp: Definition of an Event object, a combination between something like std::error_code and an solid::Any<>.
  • innerlist.hpp: A container wrapper which allows implementing bidirectional lists over a std::vector/std::deque (extensively used by the solid_frame_ipc library).
  • memoryfile.hpp: A data store with file like interface.
  • workpool.hpp: Generic thread pool.
  • dynamictype.hpp: Base for objects with alternative support to dynamic_cast
  • dynamicpointer.hpp: Smart pointer to "dynamic" objects - objects with alternative support to dynamic_cast.
  • queue.hpp: An alternative to std::queue
  • stack.hpp: An alternative to std::stack
  • algorithm.hpp: Some inline algorithms
  • common.hpp: Some bits related algorithms

Here are some sample code for some of the above tools:

Any: Sample code

	using AnyT = solid::Any<>;
	//...
	AnyT	a;
	
	a = std::string("Some text");
	
	//...
	std::string *pstr = a.cast<std::string>();
	if(pstr){
		cout<<*pstr<<endl;
	}

Some code to show the difference from boost::any:

	using AnyT = solid::Any<128>;
	
	struct A{
		uint64_t	a;
		uint64_t	b;
		double		c;
	};
	
	struct B{
		uint64_t	a;
		uint64_t	b;
		double		c;
		char		buf[64];
	};
	
	struct C{
		uint64_t	a;
		uint64_t	b;
		double		c;
		char		buf[128];
	};
	
	AnyT	a;
	//...
	a = std::string("Some text");//as long as sizeof(std::string) <= 128 no allocation is made - Any uses placement new()
	//...
	a = A();//as long as sizeof(A) <= 128 no allocation is made - Any uses placement new()
	//...
	a = B();//as long as sizeof(B) <= 128 no allocation is made - Any uses placement new()
	//...
	a = C();//sizeof(C) > 128 so new allocation is made - Any uses new
	

Event: Sample code

Create a new event category:

enum struct AlphaEvents{
	First,
	Second,
	Third,
};
using AlphaEventCategory = EventCategory<AlphaEvents>;
const AlphaEventCategory	alpha_event_category{
	"::alpha_event_category",
	[](const AlphaEvents _evt){
		switch(_evt){
			case AlphaEvents::First:
				return "first";
			case AlphaEvents::Second:
				return "second";
			case AlphaEvents::Third:
				return "third";
			default:
				return "unknown";
		}
	}
};

Handle events:

void Object::handleEvent(Event &&_revt){
	static const EventHandler<void, Object&> event_handler = {
		[](Event &_revt, Object &_robj){cout<<"handle unknown event "<<_revt<< on "<<&_robj<<endl;},
		{
			{
				alpha_event_category.event(AlphaEvents::First),
				[](Event &_revt, Object &_robj){cout<<"handle event "<<_revt<<" on "<<&_robj<<endl;}
			},
			{
				alpha_event_category.event(AlphaEvents::Second),
				[](Event &_revt, Object &_robj){cout<<"handle event "<<_revt<<" on "<<&_robj<<endl;}
			},
			{
				generic_event_category.event(GenericEvents::Message),
				[](Event &_revt, Object &_robj){cout<<"handle event "<<_revt<<"("<<*_revt.any().cast<std::string>()<<") on "<<&_robj<<endl;}
			}
		}
	};
	
	event_handler.handle(_revt, *this);
}

Creating events:

	//...
	rbase.handleEvent(alpha_event_category.event(AlphaEvents::Second));
	rbase.handleEvent(generic_event_category.event(GenericEvents::Message, std::string("Some text")));
	//...

InnerList: Sample code

MemoryFile: Sample code

solid_serialization

  • binary.hpp: Binary "asynchronous" serializer/deserializer.
  • binarybasic.hpp: Some "synchronous" load/store functions for basic types.
  • typeidmap.hpp: Class for helping "asynchronous" serializer/deserializer support polymorphism: serialize pointers to base classes.

The majority of serializers/deserializers offers the following functionality:

  • Synchronously serialize a data structure to a stream (e.g. std::ostringstream)
  • Synchronously deserialize a data structure from a stream (e.g. std::istringstream)

This means that at a certain moment, one will have the data structure twice in memory: the initial one and the one from the stream.

The solid_serialization library takes another, let us call it "asynchronous" approach. In solid_serialization the marshaling is made in two overlapping steps:

  • Push data structures into serialization/deserialization engine. No serialization is done at this step. The data structure is split into sub-parts known by the serialization engine and scheduled for serialization.
  • Marshaling/Unmarshaling
    • Marshaling: Given a fixed size buffer buffer (char*) do:
      • call serializer.run to fill the buffer
      • do something with the filled buffer - write it on socket, on file etc.
      • loop until serialization finishes.
    • Unmarshaling: Given a fixed size buffer (char*) do:
      • fill the buffer with data from a file/socket etc.
      • call deserializer.run with the given data
      • loop until deserialization finishes

This approach allows serializing data that is bigger than the system memory - e.g. serializing a data structure containing a file stream (see ipc file tutorial especially messages definition).

Sample code

A structure with serialization support:

struct Test{
	using KeyValueVectorT = std::vector<std::pair<std::string, std::string>>;
	using MapT = std::map<std::string, uint64_t>;
	
	std::string			str;
	KeyValueVectorT		kv_vec;
	MapT				kv_map;
	uint32_t			v32;
	
	template <class S>
	void serialize(S &_s){
		_s.push(str, "Test::str");
		_s.pushContainer(kv_vec, "Test::kv_vec").pushContainer(kv_map, "Test::kv_map");
		_s.pushCross(v32, "Test::v32");
	}
};

Defining the serializer/deserializer/typeidmap:

#include "solid/serialization/binary.hpp"

using SerializerT	= serialization::binary::Serializer<void>;
using DeserializerT = serialization::binary::Deserializer<void>;
using TypeIdMapT	= serialization::TypeIdMap<SerializerT, DeserializerT>;

Prepare the typeidmap:

TypeIdMapT	typemap;
typemap.registerType<Test>(0/*protocol ID*/);

Serialize and deserialize a Test structure:

	std::string		data;
	{//serialize
		SerializerT		ser(&typeidmap);
		const int		bufcp = 64;
		char 			buf[bufcp];
		int				rv;
		
		std::shared_ptr<Test>	test_ptr = Test::create();
		test_ptr->init();
		
		ser.push(test_ptr, "test_ptr");
		
		while((rv = ser.run(buf, bufcp)) > 0){
			data.append(buf, rv);
		}
	}
	{//deserialize
		DeserializerT			des(&typeidmap);
		
		std::shared_ptr<Test>	test_ptr;
		
		des.push(test_ptr, "test_ptr");
		
		size_t					rv = des.run(data.data(), data.size());
		SOLID_CHECK(rv == data.size());
	}

solid_frame

The library offers the base support for an asynchronous active object model and implementation for objects with basic support for notification and timer events.

  • manager.hpp: A synchronous, passive store of ObjectBase grouped by services.
  • object.hpp: An active object with support for events: notification events and timer events.
  • reactor.hpp: An active store of Objects with support for notification events and timer events.
  • reactorcontext.hpp: A context class given as parameter to every callback called from the reactor.
  • scheduler.hpp: A generic pool of threads running reactors.
  • service.hpp: A way of grouping related objects.
  • timer.hpp: Used by Objects needing timer events.
  • sharedstore.hpp: A store of shared object with synchronized non-conflicting read/read-write access.
  • reactorbase.hpp: Base for all reactors
  • timestore.hpp: Used by reactors for timer events support.
  • schedulerbase.hpp: Base for all schedulers.
  • objectbase.hpp: Base for all active Objects

Usefull links

solid_frame_aio

The library extends solid_frame with active objects supporting IO, notification and timer events.

  • aiodatagram.hpp: Used by aio::Objects to support asynchronous UDP communication.
  • aiostream.hpp: Used by aio::Objects to support asynchronous TCP communication.
  • aiotimer.hpp: Used by aio::Objects needing timer events.
  • aiolistener.hpp: Used by aio::Objects listening for TCP connections.
  • aiosocket.hpp: Plain socket access used by Listener/Stream and Datagram
  • aioresolver.hpp: Asynchronous address resolver.
  • aioreactorcontext.hpp: A context class given as parameter to every callback called from the aio::Reactor.
  • aioreactor.hpp: An active store of aio::Objects with support for IO, notification and timer events.

Usefull links

solid_frame_aio_openssl

The library extends solid_frame_aio with support for Secure Sockets.

solid_frame_ipc

Inter Process Communication library via Plain/Secure TCP connections and a protocol based on solid_serialization.

  • ipcservice.hpp: Main interface of the library. Sends ipc::Messages to different recipients and receives ipc::Messages.
  • ipcmessage.hpp: Base class for all messages sent through ipc::Service.
  • ipccontext.hpp: A context class given to all callbacks called by the ipc library.
  • ipcconfiguration.hpp: Configuration data for ipc::Service.

Usefull links

solid_frame_file

The library offers a specialization of frame::ShareStore for files.

  • filestore.hpp: specialization of frame::SharedStore for files with support for temporary files.
  • filestream.hpp: std::stream support
  • tempbase.hpp: Base class for temporary files: either in memory or disk files.

About

Cross-platform C++ framework for scalable, asynchronous, distributed client-server applications.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 95.5%
  • C 2.6%
  • CMake 1.5%
  • Other 0.4%