Re: [EE] Ideas about improving safety in C language

Sergio,


I don't know if it is useful to have the self-reference inside the 
descriptor, because for now the descriptor is not copiable. Indeed, all 
that the programmer interacts with is just a pointer to an opaque 
structure, which is initialized with the address of a second hidden 
variable that is the real s_string object, with header and storage area. 
The header has variable length, depending on the options used to create 
the s_string. The pointer variable is declared 'const', so it cannot be 
changed to point to another area, so it is pretty safe.


Because of that, it is also not possible to pass s_strings by value, 
only by reference.


For now, s_strings are monolithic, the descriptor and storage area are 
allocated together

I plan creating growable s_strings, perhaps with reference counting to 
allow for "copy on write", etc.


Summarizing:

The programmer has access to a const pointer to an unspecified 
structure, so he cannot mess with it (one can always cast the pointer to 
a pointer to something else, say, an array of char, but then any mishap 
cannot not be considered accidental).


I have a lot of ideas about new features. As I get some free time I will 
keep implementing them.

Now I'm thinking about the best way to implement arrays of s_strings. I 
think I have already a good solution.

Right now tt dawned on me that local/automatic s_strings will never be 
growable, because to be growable the storage area must be separated from 
the descriptor, with a pointer in the descriptor. If a local s_string 
variable ever grows, a new storage area needs to be dynamically 
allocated, then when the function returns the storage area must be 
freed, but it won't happen automatically and it is not reasonable to 
require that the programmer call a 'destructor' for each growable local 
s_string.


Cheers,

Isaac


Em 25/04/2025 08:58, smplx escreveu:
> Hi Isaac,
>
> Very good. I look forward to reading the source code when time permits.
>
> One thought, you might consider adding a field in the string 
> descriptor that is initialised to the address of the descriptor. That 
> way if the descriptor is copied then the underlying library can detect 
> it. You could use this info when changing the pointer to the payload 
> (e.g. realloc) to trap the error.
>
> A possible accidental copy would occure if the string is passed by 
> value instead of reference.
>
> Friendly Regards
> Sergio Masci
>
>
> On Wed, 23 Apr 2025, Isaac Marino Bavaresco wrote:
>
>> Hi Sergio,
>>
>> Sorry for the late reply. I was busy but wanted to comment on your 
>> message, so I saved it for later.
>>
>>
>> Em 18/04/2025 20:39, smplx escreveu:
>>>
>>>
>>> On Fri, 18 Apr 2025, Isaac Marino Bavaresco wrote:
>>>
>>>> Hi Sergio,
>>>>
>>>>
>>>> The problem you mention in (2) is not related to strings or any 
>>>> other data structure in particular, but to threads themselves. And 
>>>> for that, there are measures to solve or mitigate them. In short, 
>>>> if data is being corrupted by another thread, then the programmer 
>>>> messed up by not using a mutex or other mechanism.
>>>
>>> Hi Isaac,
>>>
>>> No you've missunderstood.
>>>
>>> Any function can be considered a form of thread but every thread 
>>> cannot be considered a function. By considering functions as threads 
>>> I am reducing the amount of work necessary to discuss the problem. 
>>> When function FOO invokes function BAR, function BAR can be 
>>> described as a thread that is launched by function FOO. FOO prepares 
>>> the context within which BAR executes and then launches it. Often 
>>> BAR will run to completion before FOO resumes, 
>>
>>
>> In this case, we have the normal function call. When BAR returns and 
>> FOO resumes, any changes made by BAR to the variables will be visible 
>> to FOO, so no problem here.
>>
>>
>> If FOO takes a mutex before calling BAR, and then BAR tries to take 
>> the mutex too, it will get the mutex, because from the system's 
>> perspective it is just the same thread taking the mutex recursively. 
>> The system will just increase the mutex's count and let BAR run. When 
>> BAR releases the mutex, the mutex's count will be decreased and when 
>> FOO releases the mutex the count will reach zero and the mutex will 
>> be really free.
>>
>>
>>> sometimes BAR will continue asynchronously and run concurrently with 
>>> FOO.
>>
>>
>> Here we have the "traditional" meaning of threads.
>>
>>
>>> If FOO invokes a mutex on a string that BAR needs to modify then a 
>>> deadlock will most likely occure.
>>
>>
>> It depends. If FOO takes the mutex before spawning BAR, then BAR 
>> tries to take the mutex and at the same time FOO is waiting for BAR 
>> to run to completion, then of course a deadlock will occur.
>>
>> In this case, perhaps it is not an application that requires threads, 
>> it most likely would be better done with just a function call as FOO 
>> will wait for BAR's completion anyway.
>>
>> In a different scenario, if FOO takes the mutex then spawns BAR, FOO 
>> will be able to finish its business with the string (or any other 
>> data) and release the mutex, then BAR will be able to do its part of 
>> the job with the string. No deadlock.
>>
>>
>>>
>>> Consider:
>>>
>>> BAR(char *str, int pos, char *stuff)
>>> {
>>>     int    len1, len2, j;
>>>
>>>     len1 = strlen(str);
>>>
>>>     len2 = strlen(stuff);
>>>
>>>     str = realloc(str, len1 + len2 + 1 - 1);
>>>
>>>     // make space at pos
>>>     for (j=len1+len2; j>pos+len2; j--)
>>>     {
>>>         str[j] = str[j-1];
>>>     }
>>>
>>>     // insert 'stuff' at pos
>>>     for (j=0; stuff[j]!='\0'; j++)
>>>     {
>>>         str[pos+j] = stuff[j];
>>>     }
>>> }
>>>
>>>
>>> FOO(char *str, char *stuff)
>>> {
>>>     int    j;
>>>
>>>     for (j=0; str[j]!='\0'; j++)
>>>     {
>>>         if (str[j] == '@')
>>>         {
>>>             BAR(str, j, stuff);
>>>
>>>             // at this point str might have been moved
>>>             // but the pointer str will not have been
>>>             // updated
>>>         }
>>>     }
>>> }
>>>
>>
>>> For the above 'str' to be safe, there needs to be a string structure 
>>> (descriptor) that is independent of the string "payload"
>>
>>
>> Well,  that was my suggestion from the beginning...
>>
>> Check my code on GitHub.
>>
>>
>>> such that the payload can be moved about in memory while the 
>>> descriptor does not move and the individual characters within the 
>>> string are always accessed using the desciptor (lingering pointers 
>>> to char are prohibited). This would require the compiler to 
>>> understand when a pointer to char is actually a pointer to a char 
>>> within a string.
>>>
>>> I have actually done some work on this in XCSB v3 but ill health has 
>>> made this a sporadic project.
>>>
>>>>
>>>> The problems I was referring to are the ones you mention in (1), 
>>>> that is, problems pertaining to string manipulation specifically. 
>>>> Standard C library functions are unsafe (they do not check maximum 
>>>> length of the destination string, for instance). It is possible to 
>>>> implement better functions that take the size of the destination 
>>>> string as an argument (they do exist, see 'strlcat', for instance), 
>>>> but the programmer can always pass a wrong size by mistake.
>>>>
>>>> What I'm suggesting is a completely new data type, that besides the 
>>>> characters themselves it stores information about maximum length, 
>>>> current used length, etc. and that is manipulated in the same way 
>>>> it is done for the present standard strings.
>>>
>>> I would strongly recommend that you break the string into 2 parts, 
>>> the string descriptor and the string payload. That way you can keep 
>>> the descriptor in a constant location while the payload hops around.
>>
>>
>> My first implementation creates strings in only one chunk, so they 
>> cannot grow beyond the maximum size they were specified upon 
>> creation. I'm thinking about the better way to implement a new 
>> version that can grow (or shrink) dynamically. I'm trying to create 
>> an implementation able to deal with both types, with the programmer 
>> choosing which type he wants to use each time.
>>
>>
>>>
>>>
>>> In C++ you could actually operload the "[]" operator but good luck 
>>> getting programmers to NOT short circuit this with "char *ptr".
>>
>>
>> I'm more interested in a C implementation, that is what I need for 
>> embedded systems. A C++ implementation seems less necessary, because, 
>> as somebody already mentioned, it already has std::string.
>>
>> Unfortunately C doesn't have function overloading. I think that some 
>> of C++'s features could be ported back to C, such as passing 
>> arguments by reference and templates, but function overloading seems 
>> too much.
>>
>>
>>>
>>>>
>>>> Of course it is not possible to have a 100% transparent and 
>>>> seamless implementation without the compiler's awareness (that is, 
>>>> implement the new strings in the language standard), but a fairly 
>>>> good solution could be created by just creating a new library.
>>>
>>> I wish you good luck
>>
>>
>> A man can dream...
>>
>>
>> Cheers,
>>
>> Isaac
>>
>>
>>>
>>> Friendly Regards
>>> Sergio Masci
>>
>> -- 
>> O software antivírus Avast realizou uma checagem de vírus neste e-mail.
>> www.avast.com
>> -- 
>> http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive
>> View/change your membership options at
>> https://mailman.mit.edu/mailman/listinfo/piclist
>>
-- 
http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive
View/change your membership options at
https://mailman.mit.edu/mailman/listinfo/piclist