-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
335 lines (278 loc) · 12.3 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
NAME
Task::MemManager - A memory allocated and manager for low level code in
Perl.
VERSION
version 0.02
SYNOPSIS
use Task::MemManager;
my $mem_manager = Task::MemManager->new(10, 1, { allocator => 'PerlAlloc' });
my $buffer = $mem_manager->get_buffer();
my $buffer_size = $mem_manager->get_buffer_size();
my $element_size = $mem_manager->get_element_size();
my $num_of_elements = $mem_manager->get_num_of_elements();
my $region = $mem_manager->extract_buffer_region($pos_start, $pos_end);
my $delayed_gc_objects = $mem_manager->get_delayed_gc_objects();
DESCRIPTION
Task::MemManager is a memory allocator and manager designed for low
level code in Perl. It provides functionalities to allocate, manage, and
manipulate memory buffers.
METHODS
new
Purpose : Allocates a buffer using a specified allocator.
Returns : A reference to the buffer.
Parameters :
- $num_of_items: Number of items in the buffer.
- $size_of_each_item: Size of each item in the buffer.
- \%opts: Reference to a hash of options. These are:
- allocator: Name of the allocator to use.
- delayed_gc: Should garbage collection be delayed?
- init_value: Value to initialize the buffer with (byte, non UTF!).
- death_stub: Function to call upon object destruction (if any).
Throws : Croaks if the buffer allocation fails.
Comments : Default allocator is PerlAlloc, which uses Perl's string functions.
Default init_value is undef ('zero' zeroes out memory,
any other byte value will initialize memory with that value).
Default delayed_gc is 0 (garbage collection is immediate).
extract_buffer_region
Usage : my $region = Task::MemManager->extract_buffer_region($pos_start, $pos_end);
Purpose : Extracts a region of the buffer.
Returns : A Perl string (null terminated) containing the region.
Parameters :
- $pos_start: The starting position of the region.
- $pos_end: The ending position of the region.
Throws : n/a
Comments : Returns undef if attempt to overrun buffer, or if $pos_start > $pos_end.
get_buffer
Usage : my $buffer = Task::MemManager->get_buffer();
Purpose : Returns the memory address of the buffer.
Returns : The memory address of the buffer as an unsigned integer.
Parameters : n/a
Throws : n/a
Comments : None.
get_buffer_size
Usage : my $buffer_size = Task::MemManager->get_buffer_size();
Purpose : Returns the size of the buffer.
Returns : The size of the buffer in bytes.
Parameters : n/a
Throws : n/a
Comments : None.
get_element_size
Usage : my $element_size = Task::MemManager->get_element_size();
Purpose : Returns the size of each element in the buffer.
Returns : The size of each element in bytes.
Parameters : n/a
Throws : n/a
Comments : None.
get_num_of_elements
Usage : my $num_of_elements = Task::MemManager->get_num_of_elements();
Purpose : Returns the number of elements in the buffer.
Returns : The number of elements in the buffer.
Parameters : n/a
Throws : n/a
Comments : None.
get_delayed_gc_objects
Usage : my $delayed_gc_objects = Task::MemManager->get_delayed_gc_objects();
Purpose : Obtains a list of objects that have delayed garbage collection.
Returns : A reference to an array of objects with delayed GC.
Parameters : n/a
Throws : n/a
Comments : None.
EXAMPLES
We will illustrate the use of these methods with multiple examples.
These will cover issues like the allocation of memory, the extraction of
regions from the buffer, constant (to the eyes of Perl) memory
allocation, delayed garbage collection, and the use of a death stub,
which is a function that is called upon object destruction and may be
used to perform e.g. logging or cleanup , operations other than freeing
the memory buffer itself. The examples are best run sequentially in a
single Perl script.
Example 1: Allocating buffers and killing them
use Task::MemManager;
## uses the default allocator PerlAlloc
my $memdeath = Task::MemManager->new(
40, 1,
{
init_value => 'zero',
death_stub => sub {
my ($obj_ref) = @_;
printf "Killing 0x%8x \n", $obj_ref->{identifier};
},
}
);
my $mem = Task::MemManager->new(
20, 1,
{
init_value => 'A',
death_stub => sub {
my ($obj_ref) = @_;
printf "Killing 0x%8x \n", $obj_ref->{identifier};
},
allocator => 'CMalloc',
}
);
printf( "%10s object is %s\n", ' mem', $mem );
$mem = Task::MemManager->new(
20, 1,
{
init_value => 'A',
death_stub => sub {
my ($obj_ref) = @_;
printf "Killing 0x%8x \n", $obj_ref->{identifier};
},
allocator => 'CMalloc',
}
);
Print the buffer objects
printf( "%10s object is %s\n", ' memdeath', $memdeath );
printf( "%10s object is %s\n", ' mem', $mem );
If you would like to kill a buffer immediately, you can undefine it:
undef $memdeath;
Attempting to under (or in general modify a constant or a Readonly)
memory buffer will kill the script. Note that these buffers can be
modified outside of Perl (including the Perl API) but not inside the
main Perl script. Such buffers are useful for keeping a constant (in
space) buffer throughout the lifetime of the script. Attempt to modify
them from within Perl, will kill the script at *runtime* uncovering the
modification attempt.
use Const::Fast; ## may also use Readonly mutatis mutandis
const my $mem_cp2 => Task::MemManager->new(
20, 1,
{
init_value => 'D',
death_stub => sub {
my ($obj_ref) = @_;
printf "Killing 0x%8x \n", $obj_ref->{identifier};
},
allocator => 'CMalloc',
}
);
undef $mem_cp2; # This will kill the script
Example 2: Extracting and inspecting a region from the buffer
First we will define a subroutine that will print the extracted region
in a nicely formated hexadecimal format.
sub print_hex_values {
my ( $string, $bytes_per_line ) = @_;
$bytes_per_line //= 8; # Default to 8 bytes per line if not provided
my @bytes = unpack( 'C*', $string ); # Unpack the string into a list of bytes
for ( my $i = 0 ; $i < @bytes ; $i++ ) {
printf( "%02X ", $bytes[$i] ); # Print each byte in hexadecimal format
print "\n"
if ( ( $i + 1 ) % $bytes_per_line == 0 )
; # Print a newline after every $bytes_per_line bytes
}
print "\n"
if ( @bytes % $bytes_per_line != 0 )
; # Print a final newline if the last line wasn't complete
}
Now let's extract the region and print it
my $region = $mem->extract_buffer_region(5, 10);
print_hex_values( $region, 8 );
Example 3: Shallow copying defers buffer deallocation
Making a shallow copy of the buffer:
my $mem_cp = $mem;
printf( "%10s object is %s\n", ' mem_cp', $mem_cp );
printf "Buffer %10s with buffer address %s\n",
'Alpha', $mem->get_buffer();
printf "Buffer %10s with buffer address %s\n",
'Alpha_copy', $mem_cp->get_buffer();
Killing the original buffer in Perl. Trying to access it after death
will lead to an error (but we intercept it in the code below)
undef $mem;
say "mem : ", ( $mem ? $mem->get_buffer() : "does not exist anymore" );
The shallow copy continues to exist, and so does the buffer region:
printf "Buffer %10s with buffer address %s\n",
'Alpha_copy', $mem_cp->get_buffer();
print_hex_values( $mem_cp->extract_buffer_region, 10 );
Example 4: Object modification and object destruction
Attempting to modify an existing buffer object, e.g. by reassiging it to
a new buffer object, will instantly free the old memory buffer, and
allocate a new buffer with new contents (this Example continues at the
end of Example 3)
$mem_cp = Task::MemManager->new(
20, 1,
{
init_value => 'D',
death_stub => sub {
my ($obj_ref) = @_;
printf "Killing 0x%8x \n", $obj_ref->{identifier};
},
allocator => 'CMalloc',
}
);
printf( "%10s object is %s\n", ' mem_cp', $mem_cp );
printf "Buffer %10s with buffer address %s\n",
'Alpha_copy after modification', $mem_cp->get_buffer();
print_hex_values( $mem_cp->extract_buffer_region, 10 );
Example 5: Fine control over garbage collection
Delayed garbage collection is useful when you want to keep a buffer
alive for a while after it goes out of scope. This is useful when you
want to transfer ownership of the memory space to an interfacing code
(e.g. C code), and don't want Perl to free the memory buffer (e.g when a
lexical variable is reassigned to a new buffer object in a loop). In
this example we will create two buffers, one without and one with
delayed garbage collection and will track when they die relative to the
end of the script. This example is entirely self-contained.
use Task::MemManager;
use strict;
use warnings;
$mem_cp = Task::MemManager->new(
20, 1,
{
init_value => 'D',
death_stub => sub {
my ($obj_ref) = @_;
printf "Killing 0x%8x \n", $obj_ref->{identifier};
},
allocator => 'PerlAlloc',
}
);
$mem_cp2 = Task::MemManager->new(
20, 1,
{
init_value => 'D',
death_stub => sub {
my ($obj_ref) = @_;
printf "Killing 0x%8x \n", $obj_ref->{identifier};
},
delayed_gc => 1,
allocator => 'CMalloc',
}
);
List the objects with delayed garbage collection
my $delayed_gc_objects = Task::MemManager->get_delayed_gc_objects();
printf "Objects with delayed GC : "
. ("0x%8x " x @$delayed_gc_objects)
. "\n", @$delayed_gc_objects;
Time the precise moment of death:
say "Undefining an object with delayed GC does not kill it!";
undef $mem_cp2;
say "End of the program - see how Perl's destroying all "
. "delayed GC objects along with the rest of the objects";
DIAGNOSTICS
There are no diagnostics that one can use. The module will croak if the
allocation fails, so you don't have to worry about error handling.
DEPENDENCIES
The module depends on the "Inline::C" module to access the memory buffer
of the Perl scalar using the PerlAPI. In addition it depends implicitly
on all the dependencies of the memory allocators it uses
TODO
Open to suggestions. A few foolish ideas of my own include: adding
further allocators and providing facilities that will *trigger* the
delayed garbage collection for a specific object, at specific time
points in a script (emulating for example Go's garbage collector).
SEE ALSO
* <https://metacpan.org/pod/Inline::C>
Inline::C is a module that allows you to write Perl subroutines in
C.
* <https://perldoc.perl.org/perlguts>
Introduction to the Perl API.
* <https://perldoc.perl.org/perlapi>
Autogenerated documentation for the perl public API.
AUTHOR
Christos Argyropoulos, "<chrisarg at cpan.org>"
COPYRIGHT AND LICENSE
This software is copyright (c) 2024 by Christos Argyropoulos.
This is free software; you can redistribute it and/or modify it under
the MIT license. The full text of the license can be found in the
LICENSE file See <https://en.wikipedia.org/wiki/MIT_License> for more
information.