Pydatacore

Fix: Correct handling of num_samples in datapool storedata method
**Full Changelog**: https://github.com/GuillaumeTrain/PyDataCore/compare/1.0.5...1.0.6

Fix: Correct handling of num_samples in data storage and chunking

- Added proper updates to `num_samples` when storing data from generators or objects.
- Ensured that `num_samples` is correctly incremented during chunk-by-chunk storage, both in RAM and file-based data storage.
- Updated the `store_data_from_object` and `store_data_from_data_generator` methods to handle large datasets, ensuring the accurate count of samples.
- Refined `ChunkableMixin` to ensure that chunk-based operations properly update and track the number of samples stored.
- Fixed potential issues with reading incomplete chunks by properly handling the remaining bytes during file-based data retrieval.
- Added more robust data handling for mixed RAM and file-based operations, with enhanced logging for debugging.

These changes address the issues with zero `num_samples` and improve the overall reliability of chunked data handling in PyDataCore.

**Full Changelog**: https://github.com/GuillaumeTrain/PyDataCore/compare/1.0.4...1.0.5

**Full Changelog**: https://github.com/GuillaumeTrain/PyDataCore/compare/1.0.3...1.0.4
Merge remote-tracking branch 'origin/main'
Ajout de la fonctionnalité de lecture de chunks spécifiques pour les données volumineuses stockées en fichier

Ajout de la méthode read_specific_chunk dans ChunkableMixin pour permettre la lecture d'un chunk spécifique directement depuis un fichier.
Amélioration de la gestion des chunks non complets (en fin de fichier) afin de garantir qu'aucune donnée n'est perdue lors de la lecture.
Validation et tests effectués pour différents types de données (TEMPORAL_SIGNAL, FREQ_SIGNAL, etc.) avec des tailles de fichier et de chunk variées.
Mise à jour du README.md pour refléter les nouvelles fonctionnalités de gestion des données volumineuses via fichiers.
Added get_data_chunk(self, data_id, chunk_index, chunk_size=1024) method to DataPool class to get a specific chunk of a data

Build failed ....
Merge remote-tracking branch 'origin/main'
Ajout de la fonctionnalité de lecture de chunks spécifiques pour les données volumineuses stockées en fichier

- Ajout de la méthode `read_specific_chunk` dans `ChunkableMixin` pour permettre la lecture d'un chunk spécifique directement depuis un fichier.
- Amélioration de la gestion des chunks non complets (en fin de fichier) afin de garantir qu'aucune donnée n'est perdue lors de la lecture.
- Validation et tests effectués pour différents types de données (TEMPORAL_SIGNAL, FREQ_SIGNAL, etc.) avec des tailles de fichier et de chunk variées.
- Mise à jour du README.md pour refléter les nouvelles fonctionnalités de gestion des données volumineuses via fichiers.
- Added get_data_chunk(self, data_id, chunk_index, chunk_size=1024) method to DataPool class to get a specific chunk of a data

**Full Changelog**: https://github.com/GuillaumeTrain/PyDataCore/compare/1.0.2...1.0.3

Initial commit for the DataPool enhancements with advanced chunk management and memory optimizations

-Fixed Data class import
- Added number of sample determination during chunk/ram storage process
- Added detailed sample base type handling (`float32`, `float64`, `int32`, `int64`) for chunk operation
- Introduced `ChunkableMixin` and `FileRamMixin` in test to support large data handling by storing and retrieving data in chunks (RAM and file-based storage)
- Implemented chunked data processing with `store_data_from_data_generator`, `read_chunked_data`, and overlapped chunking (`read_overlapped_chunked_data`)
- Added comprehensive test cases for `DataPool` and `Data` classes to validate storage methods, memory management, and handling of large datasets
- Integrated memory leak detection using `tracemalloc`
- Tested data types include `TEMPORAL_SIGNAL`, `FREQ_SIGNAL`, `FFTS`, `INTS`, `STR`, and more, with support for both in-RAM and file-based storage

**Full Changelog**: https://github.com/GuillaumeTrain/PyDataCore/compare/1.0.1...1.0.2

**Full Changelog**: https://github.com/GuillaumeTrain/PyDataCore/commits/1.0.1

** PyDataCore Project**
**Overview**
The DataPool project is designed to manage various types of data (e.g., temporal signals, frequency signals, file paths, etc.) and handle data storage in both RAM and file-based systems. This project enables dynamic registration, storage, and retrieval of data, allowing flexible handling of data chunks and memory management.

The system is capable of storing data either in RAM or as files, with support for large datasets, concurrent data access, and chunked data retrieval.

**Use Cases**
Data Registration and Storage: Register different types of data (e.g., temporal signals, frequency signals, file paths, etc.), store them either in RAM or files, and retrieve them when needed.
Data Chunking: Stream large datasets in chunks for memory-efficient processing, with both overlapped and non-overlapped chunk retrieval methods.
Concurrent Access Management: Handle multiple subscribers accessing the same data with proper acknowledgment and locking mechanisms to prevent data conflicts.
RAM and File Conversion: Dynamically convert data between RAM and file storage based on memory needs.
Data Deletion: Efficiently delete data when all subscribers have acknowledged it, with protection mechanisms in place to prevent unauthorized deletions.