[TikiWiki-commits] [Git][tikiwiki/tiki][master] [NEW] Add Markdown Importer feature

Alvin Bauma pushed to branch master at Tiki Wiki CMS Groupware / Tiki


Commits:
f316a55f by Alvin Bauma at 2026-05-12T21:38:26+02:00
[NEW] Add Markdown Importer feature
---
* [FIX] Adjust unique key length for source_key and relpath in markdown imports schema

* [REF] Add source metadata retrieval, adjust timeout, and improve error handling

* [REF] Fix magic strings

* Add orphaned entries retrieval to ImportRegistry and update import method to include orphaned data

* [REF] Enhance import logic to retry writing on previous errors for unchanged content

* [REF] Refactor SourceManager: Separate provider creation and fetching for cleaner architecture and improved error handling

* [REF] Refactor error handling in file and directory operations: Replace silent failures with explicit error handling and logging

* [REF] Refactor Markdown Purifiers: Enhance purify method documentation and add context options for improved clarity

* Refactor date parsing in tryParseJournalDate: Enhance date normalization and improve locale handling

* [REF] Refactor Source Management: Introduce SourceConfig class and update SourceManager to utilize it for improved configuration handling

* Fix CI

* [REF] Refactor Markdown Importer to Use Directory Scanning instead of ZipScanner

* [TRA] Refactor translation handling: Replace hardcoded strings with translation function calls in various importer classes and templates

* [REF] Refactor date parsing in ZipScanner to support multiple locales

* Refactor page update logic: Use Tiki's native update_page() and create_page() methods for better history management

* [REF] Improve file cleanup: Check for file existence before unlinking temporary zip files

* [ENH] Add source ID support: Implement stable identifier for markdown import sources

* [REF] Update upsert method in ImportRegistry: Return success status and handle exceptions

* [ENH] Enhance journal date parsing: Support multiple languages using IntlDateFormatter

* fix

* Refactor Markdown Importer: Remove FTP support and update source type to 'git'

* Refactor Markdown Importer: Remove FTP support, streamline Git repository handling

* [ENH] Dectect changes : Re-import the same file, making sure no new history entry is generated if the file didn changes

* [ENH] Add Markdown Importer links to admin templates for easier access

* [ENH] Add markdown:importer console command, designed for use with the Scheduler to automate recurring imports

* [NEW] Add Markdown Importer feature

- Implemented TikiMarkdownImporter class for importing Markdown files from various sources (local, FTP, GitHub/GitLab).
- Created a new template (tiki-markdown_import.tpl) for the Markdown import UI, allowing users to configure source settings and import options.
- Developed tiki-markdown_import.php to handle form submissions, manage session data, and execute import operations.
- Added support for different Markdown flavors (GFM, CommonMark, Logseq) and various import options such as roots, recursive scanning, and title strategies.

See merge request tikiwiki/tiki!8726

- - - - -


27 changed files:

- .gitignore
- db/tiki.sql
- db/tiki_convert_myisam_to_innodb.sql
- + installer/schema/20251029_markdown_importer_tiki.sql
- + lib/Importer/Markdown/Parser/CommonMarkPurifier.php
- + lib/Importer/Markdown/Parser/GfmPurifier.php
- + lib/Importer/Markdown/Parser/LogseqPurifier.php
- + lib/Importer/Markdown/Parser/MarkdownPipeline.php
- + lib/Importer/Markdown/Parser/MarkdownPurifierInterface.php
- + lib/Importer/Markdown/Registry/ImportRegistry.php
- + lib/Importer/Markdown/Scanner/DirectoryScanner.php
- + lib/Importer/Markdown/Scanner/MarkdownFileAnalyzer.php
- + lib/Importer/Markdown/Scanner/ScannerInterface.php
- + lib/Importer/Markdown/Source/FilesystemProvider.php
- + lib/Importer/Markdown/Source/GitWorkingCopyProvider.php
- + lib/Importer/Markdown/Source/SourceConfig.php
- + lib/Importer/Markdown/Source/SourceManager.php
- + lib/Importer/Markdown/Source/SourceProviderInterface.php
- + lib/Importer/Markdown/TikiMarkdownImporter.php
- lib/core/Scheduler/Task/ConsoleCommandTask.php
- lib/core/Tiki/Command/ConsoleApplicationBuilder.php
- + lib/core/Tiki/Command/MarkdownImporterCommand.php
- templates/admin/include_anchors.tpl
- templates/admin/include_wiki.tpl
- templates/tiki-importer.tpl
- + templates/tiki-markdown_import.tpl
- + tiki-markdown_import.php


The diff was not included because it is too large.


View it on GitLab: https://gitlab.com/tikiwiki/tiki/-/commit/f316a55ff1a150b4250756c89cacda5d4e34867e

-- 
View it on GitLab: https://gitlab.com/tikiwiki/tiki/-/commit/f316a55ff1a150b4250756c89cacda5d4e34867e
You're receiving this email because of your account on gitlab.com. Manage all notifications: https://gitlab.com/-/profile/notifications | Help: https://gitlab.com/help

_______________________________________________
TikiWiki-cvs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-cvs