Viadot

Added
- Added `rfc_character_limit` parameter in `SAPRFCToDF` task, `SAPRFC` source, `SAPRFCToADLS` and `SAPToDuckDB` flows
- Added `on_bcp_error` and `bcp_error_log_path` parameters in `BCPTask`
- Added ability to process queries which result exceed SAP's character per low limit in `SAPRFC` source
- Added new flow `PrefectLogs` for extracting all logs from Prefect with details
- Added `PrefectLogs` flow

Changed
- Changed `CheckColumnOrder` task and `ADLSToAzureSQL` flow to handle appending to non existing table
- Changed tasks order in `EpicorOrdersToDuckDB`, `SAPToDuckDB` and `SQLServerToDuckDB` - casting
DF to string before adding metadata
- Changed `add_ingestion_metadata_task()` to not to add metadata column when input DataFrame is empty
- Changed `check_if_empty_file()` logic according to changes in `add_ingestion_metadata_task()`
- Changed accepted values of `if_empty` parameter in `DuckDBCreateTableFromParquet`
- Updated `.gitignore` to ignore files with `*.bak` extension and to ignore `credentials.json` in any directory
- Changed logger messages in `AzureDataLakeRemove` task

Fixed
- Fixed handling empty response in `SAPRFC` source
- Fixed issue in `BCPTask` when log file couln't be opened.
- Fixed log being printed too early in `Salesforce` source, which would sometimes cause a `KeyError`
- `raise_on_error` now behaves correctly in `upsert()` when receiving incorrect return codes from Salesforce

Removed
- Removed option to run multiple queries in `SAPRFCToADLS`

Added
- Added `error_log_file_path` parameter in `BCPTask` that enables setting name of errors logs file
- Added `on_error` parameter in `BCPTask` that tells what to do if bcp error occurs.
- Added error log file and `on_bcp_error` parameter in `ADLSToAzureSQL`
- Added handling POST requests in `handle_api_response()` add added it to `Epicor` source.
- Added `SalesforceToDF` task
- Added `SalesforceToADLS` flow
- Added `overwrite_adls` option to `BigQueryToADLS` and `SharepointToADLS`
- Added `cast_df_to_str` task in `utils.py` and added this to `EpicorToDuckDB`, `SAPToDuckDB`, `SQLServerToDuckDB`
- Added `if_empty` parameter in `DuckDBCreateTableFromParquet` task and in `EpicorToDuckDB`, `SAPToDuckDB`,
`SQLServerToDuckDB` flows to check if output Parquet is empty and handle it properly.
- Added `check_if_empty_file()` and `handle_if_empty_file()` in `utils.py`

Added
- Added new connector - Outlook. Created `Outlook` source, `OutlookToDF` task and `OutlookToADLS` flow.
- Added new connector - Epicor. Created `Epicor` source, `EpicorToDF` task and `EpicorToDuckDB` flow.
- Enabled Databricks Connect in the image. To enable, [follow this guide](./README.mdexecuting-spark-jobs)
- Added `MySQL` source and `MySqlToADLS` flow
- Added `SQLServerToDF` task
- Added `SQLServerToDuckDB` flow which downloads data from SQLServer table, loads it to parquet file and then uplads it do DuckDB
- Added complete proxy set up in `SAPRFC` example (`viadot/examples/sap_rfc`)

Changed
- Changed default name for the Prefect secret holding the name of the Azure KV secret storing Sendgrid credentials

Added
- Added `func` parameter to `SAPRFC`
- Added `SAPRFCToADLS` flow which downloads data from SAP Database to to a pandas DataFrame, exports df to csv and uploads it to Azure Data Lake.
- Added `adls_file_name` in `SupermetricsToADLS` and `SharepointToADLS` flows
- Added `BigQueryToADLS` flow class which anables extract data from BigQuery.
- Added `Salesforce` source
- Added `SalesforceUpsert` task
- Added `SalesforceBulkUpsert` task
- Added C4C secret handling to `CloudForCustomersReportToADLS` flow (`c4c_credentials_secret` parameter)

Fixed
- Fixed `get_flow_last_run_date()` incorrectly parsing the date
- Fixed C4C secret handling (tasks now correctly read the secret as the credentials, rather than assuming the secret is a container for credentials for all environments and trying to access specific key inside it). In other words, tasks now assume the secret holds credentials, rather than a dict of the form `{env: credentials, env2: credentials2}`
- Fixed `utils.gen_bulk_insert_query_from_df()` failing with > 1000 rows due to INSERT clause limit by chunking the data into multiple INSERTs
- Fixed `get_flow_last_run_date()` incorrectly parsing the date
- Fixed `MultipleFlows` when one flow is passed and when last flow fails.

Added
- Added `AzureDataLakeRemove` task

Changed
- Changed name of task file from `prefect` to `prefect_date_range`

Fixed
- Fixed out of range issue in `prefect_date_range`

Changed
- bumped version