Everyone knows that each time system call gets execured, the 'errno' will be set 'errno' value of last system call that failed. So it is a good practice to store 'errno' value for further processing if any system call getting called subsequently.
For ex:
socket(...); /* do not worry about args */
printf("Trying to open socket\n");
printf("The errno was = %d\n", errno);
So what is problem with above code? Well, printf is internally write system call. If printf fails, the 'errno' value will reflect the output of printf but not the socket error.
Now consider the example. I create two sockets. One with appletalk stream socket and other arbitrary protocol with datagram socket. We know stream socket is basically for TCP and SCTP based sockets. So the one with SOCK_STREAM gives error and the arbitrary also gives error. See the differences in 'errno'.
socket(AF_APPLETALK, SOCK_STREAM, 0);
socket(200, SOCK_DGRAM, 0);
printf("%d\n", errno);
Output: 97
I modify the code back.
socket(AF_APPLETALK, SOCK_STREAM, 0);
printf("%d\n", errno);
socket(200, SOCK_DGRAM, 0);
Output: 94
You will see difference in 'errno' which confirms our observation. So it is always good practice to store 'errno' you want to catch for particular system call before you proceed further.
Question comes about thread safety. 'errno' is global variable and resides in data segment which means not thread-safe. Does glibc puts any mutex locks around it. Oh thats really horrible thing and if you are using 'errno' directly, it does not make sense even. So how 'errno' is managed? What if 'errno' in one thread is modified by system call return of other thread?
Well glibc does this by making 'errno' per thread variable. For more information on gcc per thread variable see here: http://gcc.gnu.org/onlinedocs/gcc-3.3.1/gcc/Thread-Local.html
So 'errno' modification is transparent to per thread and not across threads. Hence you can safely assume that 'errno' within a thread is localized and is completely abstracted from other threads. Consider the example:
#include <sys/types.h>
#include <sys/socket.h>
#include <errno.h>
#include <stdio.h>
#include <pthread.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>
#define MAX_THREADS 3
pthread_barrier_t t_barrier;
void* syscall_thread1(void*);
void* syscall_thread2(void*);
void* syscall_thread3(void*);
int main()
{
pthread_t tid[MAX_THREADS];
/* It is not required to include main thread. Only for illustration */
if (pthread_barrier_init(&t_barrier, NULL, MAX_THREADS+1)) {
printf("Error in initializing pthread barrier\n");
exit(1);
}
if (pthread_create(&tid[0], NULL, syscall_thread1, NULL)) {
printf("Failed to create thread instance %d\n", 1);
}
if (pthread_create(&tid[1], NULL, syscall_thread2, NULL)) {
printf("Failed to create thread instance %d\n", 2);
}
if (pthread_create(&tid[2], NULL, syscall_thread3, NULL)) {
printf("Failed to create thread instance %d\n", 3);
}
/*
* Make sure threads start simultaneously
* atleast from code point of view,
* may not be from scheduler point of view
*/
pthread_barrier_wait(&t_barrier);
/* Wait for each thread to complete */
pthread_join(tid[0], NULL);
pthread_join(tid[1], NULL);
pthread_join(tid[2], NULL);
pthread_barrier_destroy(&t_barrier);
return 0;
}
void* syscall_thread1(void *instance)
{
pthread_barrier_wait(&t_barrier);
printf("In thread = %lu\n", pthread_self());
if ((socket(AF_APPLETALK, SOCK_STREAM, 0)) == -1)
perror("socket:");
}
void* syscall_thread2(void *instance)
{
pthread_barrier_wait(&t_barrier);
printf("In thread = %lu\n", pthread_self());
if ((socket(200, SOCK_STREAM, 0)) == -1)
perror("socket:");
}
void* syscall_thread3(void *instance)
{
pthread_barrier_wait(&t_barrier);
printf("In thread = %lu\n", pthread_self());
if ((open("some_non_existing_file", O_RDWR)) == -1)
perror("open:");
}
Output:
In thread = 3079547760
socket:: Socket type not supported
In thread = 3071155056
socket:: Address family not supported by protocol
In thread = 3062762352
open:: No such file or directory
Works as expected! However order of print depends on thread scheduling by linux scheduler. We may have to compile glibc with _REENTRANT option to achieve per thread 'errno' storage.
How about digging glibc code for a while?
/usr/include/errno.h says 'errno' is per-thread variable.
#ifdef _ERRNO_H
/* Declare the `errno' variable, unless it's defined as a macro by
bits/errno.h. This is the case in GNU, where it is a per-thread
variable. This redeclaration using the macro still works, but it
will be a function declaration without a prototype and may trigger
a -Wstrict-prototypes warning. */
#ifndef errno
extern int errno;
#endif
another snippet under /usr/include/i386-linux-gnu/bits/errno.h
# if !defined _LIBC || defined _LIBC_REENTRANT
/* When using threads, errno is a per-thread value. */
# define errno (*__errno_location ())
# endif
# endif /* !__ASSEMBLER__ */
#endif /* _ERRNO_H */
basically errno_location() function gets address of global 'errno' variable. As per the article in here: http://pauillac.inria.fr/~xleroy/linuxthreads/faq.html,
"Thus, for programs not linked with LinuxThreads, defining _REENTRANT makes no difference w.r.t. errno processing. But LinuxThreads redefines __errno_location() to return a location in the thread descriptor reserved for holding the current value of errno for the calling thread. Thus, each thread operates on a different errno location."
which means that each thread gets its own copy of 'errno'.
Small search in cscope for errno_location() leads to hurd/hurd/threadvar.h which says it all. What it means need to dig still further which even I am doing :-). The value seems to be obtained from some offset from thread stack pointer (just assuming).
/* Return the location of the value for the per-thread variable with index
INDEX used by the thread whose stack pointer is SP. */
extern unsigned long int *__hurd_threadvar_location_from_sp
(enum __hurd_threadvar_index __index, void *__sp);
_HURD_THREADVAR_H_EXTERN_INLINE unsigned long int *
__hurd_threadvar_location_from_sp (enum __hurd_threadvar_index __index,
void *__sp)
{
unsigned long int __stack = (unsigned long int) __sp;
return &((__stack >= __hurd_sigthread_stack_base &&
__stack < __hurd_sigthread_stack_end)
? __hurd_sigthread_variables
: (unsigned long int *) ((__stack & __hurd_threadvar_stack_mask) +
__hurd_threadvar_stack_offset))[__index];
}
/* Return the location of the current thread's value for the
per-thread variable with index INDEX. */
extern unsigned long int *
__hurd_threadvar_location (enum __hurd_threadvar_index __index) __THROW
/* This declaration tells the compiler that the value is constant
given the same argument. We assume this won't be called twice from
the same stack frame by different threads. */
__attribute__ ((__const__));
_HURD_THREADVAR_H_EXTERN_INLINE unsigned long int *
__hurd_threadvar_location (enum __hurd_threadvar_index __index)
{
return __hurd_threadvar_location_from_sp (__index,
__thread_stack_pointer ());
}
Conclusion is glibc marks 'errno' as per thread variable and it is safe to access 'errno' across threads. Any comments or suggestions are greatly welcome
Thanks to wonderful book Linux System Programming by Robert Love through which I came to know that 'errno' is thread safe.
For ex:
socket(...); /* do not worry about args */
printf("Trying to open socket\n");
printf("The errno was = %d\n", errno);
So what is problem with above code? Well, printf is internally write system call. If printf fails, the 'errno' value will reflect the output of printf but not the socket error.
Now consider the example. I create two sockets. One with appletalk stream socket and other arbitrary protocol with datagram socket. We know stream socket is basically for TCP and SCTP based sockets. So the one with SOCK_STREAM gives error and the arbitrary also gives error. See the differences in 'errno'.
socket(AF_APPLETALK, SOCK_STREAM, 0);
socket(200, SOCK_DGRAM, 0);
printf("%d\n", errno);
Output: 97
I modify the code back.
socket(AF_APPLETALK, SOCK_STREAM, 0);
printf("%d\n", errno);
socket(200, SOCK_DGRAM, 0);
Output: 94
You will see difference in 'errno' which confirms our observation. So it is always good practice to store 'errno' you want to catch for particular system call before you proceed further.
Question comes about thread safety. 'errno' is global variable and resides in data segment which means not thread-safe. Does glibc puts any mutex locks around it. Oh thats really horrible thing and if you are using 'errno' directly, it does not make sense even. So how 'errno' is managed? What if 'errno' in one thread is modified by system call return of other thread?
Well glibc does this by making 'errno' per thread variable. For more information on gcc per thread variable see here: http://gcc.gnu.org/onlinedocs/gcc-3.3.1/gcc/Thread-Local.html
So 'errno' modification is transparent to per thread and not across threads. Hence you can safely assume that 'errno' within a thread is localized and is completely abstracted from other threads. Consider the example:
#include <sys/types.h>
#include <sys/socket.h>
#include <errno.h>
#include <stdio.h>
#include <pthread.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>
#define MAX_THREADS 3
pthread_barrier_t t_barrier;
void* syscall_thread1(void*);
void* syscall_thread2(void*);
void* syscall_thread3(void*);
int main()
{
pthread_t tid[MAX_THREADS];
/* It is not required to include main thread. Only for illustration */
if (pthread_barrier_init(&t_barrier, NULL, MAX_THREADS+1)) {
printf("Error in initializing pthread barrier\n");
exit(1);
}
if (pthread_create(&tid[0], NULL, syscall_thread1, NULL)) {
printf("Failed to create thread instance %d\n", 1);
}
if (pthread_create(&tid[1], NULL, syscall_thread2, NULL)) {
printf("Failed to create thread instance %d\n", 2);
}
if (pthread_create(&tid[2], NULL, syscall_thread3, NULL)) {
printf("Failed to create thread instance %d\n", 3);
}
/*
* Make sure threads start simultaneously
* atleast from code point of view,
* may not be from scheduler point of view
*/
pthread_barrier_wait(&t_barrier);
/* Wait for each thread to complete */
pthread_join(tid[0], NULL);
pthread_join(tid[1], NULL);
pthread_join(tid[2], NULL);
pthread_barrier_destroy(&t_barrier);
return 0;
}
void* syscall_thread1(void *instance)
{
pthread_barrier_wait(&t_barrier);
printf("In thread = %lu\n", pthread_self());
if ((socket(AF_APPLETALK, SOCK_STREAM, 0)) == -1)
perror("socket:");
}
void* syscall_thread2(void *instance)
{
pthread_barrier_wait(&t_barrier);
printf("In thread = %lu\n", pthread_self());
if ((socket(200, SOCK_STREAM, 0)) == -1)
perror("socket:");
}
void* syscall_thread3(void *instance)
{
pthread_barrier_wait(&t_barrier);
printf("In thread = %lu\n", pthread_self());
if ((open("some_non_existing_file", O_RDWR)) == -1)
perror("open:");
}
Output:
In thread = 3079547760
socket:: Socket type not supported
In thread = 3071155056
socket:: Address family not supported by protocol
In thread = 3062762352
open:: No such file or directory
Works as expected! However order of print depends on thread scheduling by linux scheduler. We may have to compile glibc with _REENTRANT option to achieve per thread 'errno' storage.
How about digging glibc code for a while?
/usr/include/errno.h says 'errno' is per-thread variable.
#ifdef _ERRNO_H
/* Declare the `errno' variable, unless it's defined as a macro by
bits/errno.h. This is the case in GNU, where it is a per-thread
variable. This redeclaration using the macro still works, but it
will be a function declaration without a prototype and may trigger
a -Wstrict-prototypes warning. */
#ifndef errno
extern int errno;
#endif
another snippet under /usr/include/i386-linux-gnu/bits/errno.h
# if !defined _LIBC || defined _LIBC_REENTRANT
/* When using threads, errno is a per-thread value. */
# define errno (*__errno_location ())
# endif
# endif /* !__ASSEMBLER__ */
#endif /* _ERRNO_H */
basically errno_location() function gets address of global 'errno' variable. As per the article in here: http://pauillac.inria.fr/~xleroy/linuxthreads/faq.html,
"Thus, for programs not linked with LinuxThreads, defining _REENTRANT makes no difference w.r.t. errno processing. But LinuxThreads redefines __errno_location() to return a location in the thread descriptor reserved for holding the current value of errno for the calling thread. Thus, each thread operates on a different errno location."
which means that each thread gets its own copy of 'errno'.
Small search in cscope for errno_location() leads to hurd/hurd/threadvar.h which says it all. What it means need to dig still further which even I am doing :-). The value seems to be obtained from some offset from thread stack pointer (just assuming).
/* Return the location of the value for the per-thread variable with index
INDEX used by the thread whose stack pointer is SP. */
extern unsigned long int *__hurd_threadvar_location_from_sp
(enum __hurd_threadvar_index __index, void *__sp);
_HURD_THREADVAR_H_EXTERN_INLINE unsigned long int *
__hurd_threadvar_location_from_sp (enum __hurd_threadvar_index __index,
void *__sp)
{
unsigned long int __stack = (unsigned long int) __sp;
return &((__stack >= __hurd_sigthread_stack_base &&
__stack < __hurd_sigthread_stack_end)
? __hurd_sigthread_variables
: (unsigned long int *) ((__stack & __hurd_threadvar_stack_mask) +
__hurd_threadvar_stack_offset))[__index];
}
/* Return the location of the current thread's value for the
per-thread variable with index INDEX. */
extern unsigned long int *
__hurd_threadvar_location (enum __hurd_threadvar_index __index) __THROW
/* This declaration tells the compiler that the value is constant
given the same argument. We assume this won't be called twice from
the same stack frame by different threads. */
__attribute__ ((__const__));
_HURD_THREADVAR_H_EXTERN_INLINE unsigned long int *
__hurd_threadvar_location (enum __hurd_threadvar_index __index)
{
return __hurd_threadvar_location_from_sp (__index,
__thread_stack_pointer ());
}
Conclusion is glibc marks 'errno' as per thread variable and it is safe to access 'errno' across threads. Any comments or suggestions are greatly welcome
Thanks to wonderful book Linux System Programming by Robert Love through which I came to know that 'errno' is thread safe.
No comments:
Post a Comment