problems to be solved

this site having very generic problem statements for design and develop the solution.

Networking

Basic networking knowledge.
DNS:
DHCP:
TCP/IP stack :
TCP/UDP header:
Unicast :
Multicast :
Broadcast :
Socket program:  https://www.softprayog.in/programming/interprocess-communication-using-unix-domain-sockets

IP and its classes:
Example programs:
1. file transfer using socket
2. broadcast a message in LAN using socket
3.  select, epoll with socket

https://www.tenouk.com/Module39.html 

Android

Intent:
Activity:
Service:
Binder:
JNI:
Android Make:

Linux process and IPC :

Process:
  fork()
  exec()
  PCB?
 
IPC:
1. Pipes

              Pipe Always unidirectional, used between parent and child communication.
Data written to the write end of pipe can be read from read end of pipe.

int fd[2];

Pipe created  two file descriptors using pipe system call. i.e pipe(fd); returns -1 on failure.

fd[0] - read, fd[1] - write its similar for file descriptors regular read()/write() api's used for read and write.

 - Data written to the write end of the pipe is Buffered by the Kernel until it is read from the read end of the pipe.

- If process attempt to read from empty pipe then read will block until data available.

- If process attempt to write on a non empty(full data in pipe) write api will block until sufficient data  has been read from pipe to allow the write to complete.

- Non blocking I/O possible by setting the system flag O_NONBLOCK using fcntl system api.

- The communication channel provided by pipes is a byte stream, there is no concept of message   boundaries.

- If write end fd[1] closed and read end try to read it will see EOF and read returns 0.

- If read end fd[0] closed and write will cause the SIGPIPE signal will generate to calling process, if process ignore the signel write will fail with the EPIPE error.

 -  close api should be used to close the file descriptors close(fd[0]/fd[1]).

- the capacity of the pipe is same as the system page size. 

- Threads can communicate with pipes without synchronization.

Disadvantage: its works only related process i.e fd file descriptors should be within a process.

---------

how the kernel implements pipes (at a high level).

• Linux has a VFS (virtual file system) module called pipefs, that gets mounted in kernel space during boot
• pipefs is mounted alongside the root file system (/), not in it (pipe’s root is pipe:)
• pipefs cannot be directly examined by the user unlike most file systems
• The entry point to pipefs is the pipe(2) syscall
• The pipe(2) syscall is used by shells and other programs to implement piping, and just creates a new file in pipefs, returning two file descriptors (one for the read end, opening using O_RDONLY, and one for the write end, opened using O_WRONLY)
• pipefs is stored using an in-memory file system

---------

2. Fifo - named pipes:

     The disadvantage of pipe is within a process fd will be used for read write, to over come that named pipes i.e to communicate with unrelated process by providing the name to the pipe.

    Command line api's to create the mkfifo, mknod.

Ex1: mkfifo file_name will create the file of type pipe.

Ex2: mknod filename p create the file with type pipe, here p denotes the pipe type file. as mknode used for other types like char file creation etc.

Main features: FIFO is the name within a file system, it can be open just like normal files, as same as read/write for reading writing.

- Blocking on fifo is open for read but it will block until write writes to it and vice versa for write.

 - cab be unblock using O_NONBLOCKING flag in open system call.

Disadvantage: pipes follows strict FIFO behaviour.

3. System V message queues(MQ):
      Message Queue can behave similar to pipes but FIFO order can be changed.
      MQ having system wide limit which can be seen with ipcs -l command '
       ------ Messages Limits --------
max queues system wide = 32000
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384
      MQ is a sequence of messages each which has two parts:
      1. the payload, which is an array bytes.
      2. A type, positive number, will be helpful to flexible retrieval(based on need, not like FIFO order to read).
    To create the message queue, we need ipc key, key can be created using ftok() api.
    #include <sys/types.h>
#include <sys/ipc.h>

key_t ftok (const char *pathname, int proj_id);

      pathname must be existing accessible file, content of file is immaterial.
      proj_id  lowest 8 bits of project id will be used and it should be not zero.

     msgget: system call gets the message queue identifier for the given key.
     #include <sys/msg.h>
        int msgget (key_t key, int msg_flags);

         key : is received from ftok. some times special key IPC_PRIVATE used for private mq.details
         msg_flags: IPC_CREATE for new key, if it order with IPC_EXCL for already existing q with permissions with octal values.
     msget returns integer identifier, will be used for next send receive and control message api's.

    msgctl: with this can do control operations on message queue.
      int msgctl (int msqid, int cmd, struct msqid_ds *buf);
      msqid: qid returned from msgget.
      cmd: cammand can be IPC_RMID, IPC_STAT and IPC_SET. will use IPC_RMID to remove message queue once its usage finishes.more on msgq buffer info

    msgsnd : sending messages to message queue.
int msgsnd (int msqid, const void *msgp, size_t msgsz, int msgflg);
       msgp: message buffer  
       msgsz : message size
       msgflg: it can be 0, or IPC_NOWAIT.
msgrcv: receiving message from message queue.
ssize_t msgrcv (int msqid, void *msgp, size_t msgsz, long msgtyp, int msgflg);
    

       msgp: message buffer    

       msgsz : message size

        msgtype : type of message to be retrieved.

       msgflg: it can be 0, or IPC_NOWAIT.

Example code :

//----keep below data in queue.h----------

#include <stdio.h>
#include <sys/ipc.h>
#include <sys/msg.h>
#include <stdlib.h>
#include <string.h>

#define projid 123
#define PathName  "/tmp/myqueue" /* any existing, accessible file would do */
#define MsgLen    4
#define MsgCount  6

typedef struct {
  long type;                 /* must be of type long */
  char payload[MsgLen + 1];  /* bytes in the message */
} queuedMessage;

void report_and_exit(const char* msg) {
  perror(msg);
  exit(-1); /* EXIT_FAILURE */
}

//------------------------------------------

// Inserver server.cpp:

#include "queue.h"

int main() {
  key_t key = ftok(PathName, ProjectId);
  if (key < 0) report_and_exit("couldn't get key...");

  int qid = msgget(key, 0666 | IPC_CREAT);
  if (qid < 0) report_and_exit("couldn't get queue id...");

  char* payloads[] = {"msg1", "msg2", "msg3", "msg4", "msg5", "msg6"};
  int types[] = {1, 1, 2, 2, 3, 3}; /* each must be > 0 */
  int i;
  for (i = 0; i < MsgCount; i++) {
    /* build the message */
    queuedMessage msg;
    msg.type = types[i];
    strcpy(msg.payload, payloads[i]);

    /* send the message */
    msgsnd(qid, &msg, sizeof(msg), IPC_NOWAIT); /* don't block */
    printf("%s sent as type %i\n", msg.payload, (int) msg.type);
  }
  return 0;
}

//client.cpp:

#include "queue.h"

int main() {
  key_t key= ftok(PathName, ProjectId); /* key to identify the queue */
  if (key < 0) report_and_exit("key not gotten...");

  int qid = msgget(key, 0666 | IPC_CREAT); /* access if created already */
  if (qid < 0) report_and_exit("no access to queue...");

  int types[] = {3, 1, 2, 1, 3, 2}; /* different than in sender */
  int i;
  for (i = 0; i < MsgCount; i++) {
    queuedMessage msg; /* defined in queue.h */
    if (msgrcv(qid, &msg, sizeof(msg), types[i], MSG_NOERROR | IPC_NOWAIT) < 0)
      puts("msgrcv trouble...");
    printf("%s received as type %i\n", msg.payload, (int) msg.type);
  }

  /** remove the queue **/
  if (msgctl(qid, IPC_RMID, NULL) < 0)  /* NULL = 'no flags' */
    report_and_exit("trouble removing queue...");

  return 0;
}

Problem with the pipes, fifos and message queues, the work involved in sending data from one process to another is like this. Process P1 makes a system call to send data to Process P2. The message is copied from the address space of the first process to the kernel space during the system call for sending the message. Then, the second process makes a system call to receive the message. The message is copied from the kernel space to the address space of the second process. 

The shared memory mechanism does away with this copying overhead.

4. shared memory
Fasted IPC is shared memory.
 Shared memory segment created by the kernel and mapped to the data segment of the address space of the requested process.


To use system V IPC like above message queues, need system v ipc key, which can get with ftok api.

#include <sys/types.h>
#include <sys/ipc.h>
key_t ftok (const char *pathname, int proj_id);

shmget: it gets the shared memory segment associated with key.

#include<sys/shm.h>
int shmid = shmget(key_t key, size_t size, int shmflgs); 
key - obtained with ftok or it can be IPC_PRIVATE.
size - size of the shared memory segment to be created, its a multiples of PAGE_SIZE.
shmflgs - IPC_CREATE | 0660 permissions.

shmat: With this calling process can attach the shared memory  segment with shmid got from shmget.

void *shmat (int shmid, const void *shmaddr, int shmflg);

 shmid - from shmget return value.
shmaddr - it can be null or process can specify the address at which the shared meory segment should be attached with shmaddr.
shmflg - SHM_RDONLY there are more flags..
 On error returns 1, on success returns shared memory address.

shmdt: with this detach the shared memory segment from calling process  address space.

int shmdt (const void *shmaddr);

shmaddr - shared memory address space returned at shmat

 on return -1 for error, 0 on success.

shmctl: to control the share memory segment.

int shmctl (int shmid, int cmd, struct shmid_ds *buf);

shmid - from shmget.
cmd - IPC_STAT, IPC_RMID etc.
shmid_ds is a structure to control shared meory segment params.

return -1 on error


5. Shared File
      

6. semaphore

7. pthreads

8. mutex

9. conditional variables

10. unix socket for local IPC 

     https://www.softprayog.in/programming/interprocess-communication-using-unix-domain-sockets

system calls:

Kernel and its internals:

Driver programming:

c++11 features

C++11 Features:

1. Rvalue references and move constructors
 

Lvalue	R value	Exam
lvalue is anything whose address is accessible. It means we can take address of lvalue using & operator	Rvalue is anything that is not lvalue. cannot take address of rvalue.	int x(lvalue) = 10(r value);  int z (lvalue) = (x+1) (r value); here we cannot take address of (x+1) but we can take z as it is l value int y = &z;
	It also doesn’t persist beyond the single expression.	int getVal() {      int val = 10;      return val; value of return is temporary as after it returns it won't persist } int y = getval(); getval is rvalue which will not  get the address
	Rvalues of built-in data type is immutable	(x+1) = 10; //compiler  error cannot modify rvalue Getval() = 8; // compiler error cannot modify rvalue
	Rvalue of user defined data type is not immutable	Class abc{        int x;        public:            abc() :x(0) {}             add(int val){ this.x +=val;} } abc getObj(){       return abc(); } Int main() {        abc * = & getObj(); //compiler error        abc().add(10); // it increments value but add method here is r value. }
Lvalue reference	R value reference
A Reference variable is an alias that always points to a an existing variable i.e	rvalue references can do what lvalue references fails to do i.e. a rvalue reference can refer to rvalues.	lvalue reference: int x =10; int &z = x;// Works, but int &z = (x+2) //won’t work. As l value refers l values only.  rvalue reference: int &&z=(x+2); // is valid  as now z is rvalue reference with && is syntax int &&y = getval(); // also works

Move constructor: In addition to default, copy and copy assignment operator constructor c++11 introduced move constructor and move assignment operator constructor.
default, copy constructor, copy assignment operator constructors takes lvalue reference as a argument and just copy from source object to destination object means both the copies are present in source and destination objects. Where as in move constructor will take rvalue as a input and will move from source to destination object so source object empty after moving.
If user not implements the implicit defined move special member functions move their sub objects and data members in member wise.

Move constructor doesn’t allocate resources, shift the control of memory and passed object assigned to NULL.	Move assignment operator first check the passed object and this object then constructor doesn’t allocate resources, shift the control of memory, delete passed object data pointers  and passed object assigned to NULL.
abc::abc(abc&& obj);	abc& abc::operator=(abc&& obj)
abc::abc(abc &&obj) {       m_data = obj.m_data; //move       obj.m_data = NULL; //reset passed object data to null }	abc& abc::operator=(abc &&obj) {      If(&obj != this) //check before move       m_data = obj.m_data; //move       delete obj.m_data; //delete passed object       obj.m_data = NULL; //reset its object data to null       return *this; return new moved object }

2. constexpr – Generalized constant expressions
    

Constexpr feature is for performance improvement of program by doing computations at compile time instead of run time.

 Ex1: constexpr rand(int x, int y) // here constexpr tells to compute the value at compile time.

       {

              return x+ y+20;

        }

       int val = rand(20, 30);

Ex2:

       int arrsize1(int x)                                 constexpr Int arrsize2(int x)

        {                                                              {

                 return x+1;                                           return x+1;

          }                                                             }

 int arr[arrsize1(20)]; //error as arrsize1() function will compute at run time

 int arr[arrsize2(20)]; //error as arrsize2() function will compute at compile time bcz of constexpr

Note: constexpr function must follow below conditions
       -  not a virtual function
       -  return type and its param should be in literal
       -  constexpr must refere global variable.
       -  constexpr function can call only other constexpr function not simple function
      - inside function not allow post, pre increment(++, --) expressions.
constexpr vs inline functions: inline means compiler to expand at at compile time, save time in function call overhead and evaluation done at run time, where as constexpr is evaluated at run time.
constexpr vs const:   constexpr is mainly for optimization while const is for practically const objects like value of Pi(3,14).  
valid, invalid places to use constexpr

3. Uniform initialization
    - It allows the consistency syntax to initialize the variables and class objects, initialization uses {} open close curly braces. initializer more
    - typename variable{arg1, arg2, arg3, ...}
    Ex:
            -. int x{}; //uninitialized built in type
            - .  int x{1}; //initialized built in type
            -. int arr[]{1,2,3,4}; //aggregate initializing
            - 
int* pi = new int[5]{ 1, 2, 3, 4, 5 }; //initializing dynamically allocated array

- class A { //initializing the class array member variable
int arr[3];
         public:
A(int x, int y, int z):arr{x,y,z};
               {} 
};
- std::vector<int> x{1,2,3} //Initializing the STL container
     
4. Type inference (auto)  auto
       auto keyword can be used in place of variable type to tell the compiler infer the variable type from the variable type is called type inference/type deduction.
       auto keyword is very useful to reducing the verbosity of the code.i.e
      for (std::vector <int>::iterator it = v.begin(); it!= v.end(); ++it) can be written
      for(auto it = v.begin(); it !=v.end(); ++it) // more readable as auto is simple to  write
   Notes:
      - auto key work cannot be used with function parameters ex: int fun(auto x, auto y) // not allowed
      - trailing return syntax Ex: auto func(int x, int y) -> int // it means return type is int
   
5. Range-based for loop  range based for loop
     easy iteration over the range of  elements, STL containers contain begin(), end() range of elements will be iterate easily with range based for loop.
   syntax: for(initialize starting value from range: range expression)
          ex: for(auto it : vec) // here vec is vector and it is iterator will assigned form begin() to end () element.
     for_each(): UnaryFunction for_each( InputIt first, InputIt last, UnaryFunction f );
                 
     
        #include <iostream>
#include<vector>
#include<algorithm>
using namespace std;
class Sum{
   
  public:
       Sum():_sum(0){}
       int _sum;
       void operator()(int n){
           _sum += n;
       }
};
int main()
{
   cout << "Hello World" << endl;
   std::vector<int>vec{1,2,3,4,5,6,8};
   auto f = [](const int n){
       cout<<" "<<n;
   };
   auto f2 = [](int& n){
       n++;
   };
   for_each(vec.begin(), vec.end(),f);
   for_each(vec.begin(), vec.end(),f2);
   cout<<"\n";
   for_each(vec.begin(), vec.end(),f);
   Sum s = for_each(vec.begin(), vec.end(),Sum());
   cout<<"\n"<<s._sum;;
   return 0;
}

6. Lambda functions and expressions
    Anonymous / unnamed (no name to the function)/ lambda expressions are small snippet of code inside other functions.
    Syntax :      

 [ captureClause ] ( parameters ) -> returnType
   {
      statements;
   }

   capturescluse, parameters can be empty and return type is optional by

   default return type treated as auto.

   Ex: auto f = [] (){ cout<<"Hello Lamdas "<<endl}
 

The difference between the usage of functions and lambda functions boils down to two points:

• We cannot overload lambdas.
• A lambda function can capture local variables.

• A lambda should be short and concise.
• A lambda should be self-explanatory, especially since it does not have a name.

7. Explicit overrides and final
    Explicit: Explicit function specifier control the unwanted implicit type conversion.
    - It can only be used in declaration of class constructor within the class declaration.
    class abc{
               public:
                      abc(){}
                     abc(int val) : m_val(val){}
              private:
                     int m_val;
    };
    int main()
    {
          abc a=10;// compiler error
    }
   Override: Keyword override will make the compiler to check the base class to see if there is a virtual function with this exact signature. And if there is not, the compiler will show an error. 
    Final: final Sometimes you don’t want to allow derived class to override the base class’ virtual function. C++ 11 allows built-in facility to prevent overriding of virtual function using final specifier.
      
8. Null pointer constant (nullptr)
nullptr is a keyword that can be used at all places where NULL is expected. Like NULL, nullptr is implicitly convertible and comparable to any pointer type. Unlike NULL, it is not implicitly convertible or comparable to integral types

9. Strongly typed enumerations
     enum classes / scoped enumeration which makes the enums strongly scoped and typed.
 Ex:

enum class Color // "enum class" defines this as a scoped enumeration instead of a standard enumeration

    {

        RED, // RED is inside the scope of Color

        BLUE

    };

    enum class Fruit

    {

        BANANA, // BANANA is inside the scope of Fruit

        APPLE

    };

    Color color = Color::RED; // note: RED is not directly accessible any more, we have to use Color::RED

    Fruit fruit = Fruit::BANANA; // note: BANANA is not directly accessible any more, we have to use Fruit::BANANA

    if (color == fruit) // compile error here, as the compiler doesn't know how to compare different types Color and Fruit

        std::cout << "color and fruit are equal\n";

    else

        std::cout << "color and fruit are not equal\n";

   
10. Multithreading memory model
11. Thread-local storage
12.  Explicitly defaulted and deleted special member functions
13. Static assertions
14. General-purpose smart pointers
      

         Smart pointers are introduced to prevent the memory leaks

Unique_ptr<>:unique_ptr<> objects wraps around the raw pointer and its responsible for its life time. When these objects is destructed/out of scope then in its destructor it deletes associated raw pointer.

- A unique_ptr object is always the unique owner of associated raw pointer, we cannot copy only it’s a movable.

- It will be used when u want the single ownership of the resource 

- unique_ptr has its -> and * operator overloaded, so we can use similar like normal pointers.
   unique_ptr<int>uptr(new int);//unique pointer with raw pointer
    *uptr = 100;
    cout<<"pointer conent: "<<*uptr<<endl;
    int *rptr = new int();
    *rptr = 23;
    unique_ptr<int>uptr1(rptr);//from existing raw pointer
    cout<<"pointer conent: "<<*uptr1<<endl;
    unique_ptr<int>emptuptr;//empty unique pointer
    if(emptuptr == nullptr)
    {
        //reset will delete the associated raw pointer and unique_ptr make empty
        uptr1.reset();// resetting the unique pointer
        cout<<"unptr is empty if no raw pointer assined\n";
    }
   // unique_ptr<int>cuptr=uptr; //compilation error cannot assign
//    emptuptr = upter;// compilation error  cannot assign
    unique_ptr<int>muptr = std::move(uptr);
    if(uptr == nullptr)
    cout<<"Empty uptr after move\n";
    if(muptr != nullptr)
    cout<<"Not an empty after move got ownership: "<<*muptr<<endl;
//relese
    int *relptr = muptr.release(); //returns the associated raw pointer
    cout<<"raw pointer content: "<<*relptr<<endl;

shared_ptr<>:  is smart enough to automatically delete the associated pointer when its not used anywhere. Thus helps us to completely remove the problem of memory leaks and dangling Pointers.
- Shared_ptr will be used when shared ownership of resource needed
- It is a reference counting ownership model.
- Each shared pointer internally points to two memory locations i.e 1. pointer to the object 2. pointer to the control data that is used for reference counting.
Ex: shared_ptr<int>sptr(new int());
     1. pointer to the int object, 2. memory of reference counting now count value is 1.
- raw pointer object will be deleted once the reference count becomes zero(0).
- sptr.use_count() will give the reference count value now that is 1 as per above sptr object as not shared to other pointers.
- custom delete as a function pointer we should pass when you make shared pointer scenarios like array of object shown below example
 - custom delete can be lamda function, functor object as well example
 - shared pointer spport only -> , *.
 - shared pointer not support +, -, ++, --, [] operators.
int *ptr = new int();// raw pointer
  *ptr = 10;
  shared_ptr<int>sptr(ptr);//shared pointer from raw pointer
  cout<<"\nsptr count "<<sptr.use_count()<<" "<<*sptr<<endl;
  shared_ptr<int>p2(sptr);
  shared_ptr<int>p3 = p2;// compiler error not allowed assignment

  sptr.reset();//reset will decrease reference count by 1 and delete if count is 0.
  cout<<"\n refene count"<<sptr.use_count()<<" p2 "<<p2.use_count()<<endl;
    sptr.reset(new int[12]); it won't delete arra like delete [] so needed custom delete shown below
  cout<<"\n refene count"<<sptr.use_count()<<" p2 "<<p2.use_count()<<endl;
   void customDel(int x){
          delete[] x; // to delete array [] required so custom delete needs to be written
   }

  shared_ptr<int>p(new int[12], customDel);
   cout<<"\n refene count"<<sptr.use_count()<<" p "<<p.use_count()<<endl;
   sptr = nullptr; // other way to reset using null pointer same count will decrease by 1 and delete if count is 0
    cout<<"\n refene count"<<sptr.use_count()<<" p2 "<<p2.use_count()<<endl;

Notes for shared pointer:carefull
 - Try not to use same raw pointer for more than one shared_ptr objects.
    Ex: int *p = new int();
          shared_ptr<int>sp1(p);{
          shared_ptr<int>sp2(p);// wrong u not aware if sp2 goes out of scope then raw pointer deleted when sp1 goes out of scope then its a dangling pointer
         }
 - Don't create shared pointer from stack memory
      Ex: int x= 10;
             shared_ptr<int>spt2(&x); // &x is stack address when shared pointer goes out of scope then deleting stack memory using delete will crash the program

Week_ptr<>: weekpointer  allows sharing but not owns the object i.e not maintain reference count.
- circular dependency of shared object problem can be solved using the week pointer
Ex: Class A, class B objects maintain the class object in each other then reference count never comes to 0 if we use shared pointer.

15. Extensible random number facility
16. Wrapper reference std::ref()
17. Uniform method for computing the return type of function objects
18. Lambdas
19. non-member begin() and end()
20.