Re: Bulk insert content
Michael Emmerich via opencms-dev <[email protected]> Fri, 17 Oct 2025 09:39:28 +0200
| Newsgroups | gmane.comp.cms.opencms.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format.
--===============7171036632105921868==
Content-Type: multipart/alternative;
boundary="------------9jTrFn3rxbG1AIdBBnI6YReH"
Content-Language: en-US
This is a multi-part message in MIME format.
--------------9jTrFn3rxbG1AIdBBnI6YReH
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Hello Atle,
first of all — *inserting data directly into the OpenCms database is
strongly discouraged*. Doing so can easily corrupt your system.
The correct and safe way to migrate or import data is to *create an
importer* that uses the *OpenCms API* to generate content based on the
data from your existing system. Fortunately, this is not as complicated
as it sounds.
Step 1: Export Data from Your Current System
Before you can import anything into OpenCms, determine how you can
*export* your articles from your current system, and in what *format*
(e.g., XML, JSON, CSV).
Each article should ideally be exported as a separate file — for
example, one XML file per article — or retrieved directly via an
interface (API).
Step 2: Import Process Overview
For each article, your import process will roughly follow these steps:
1.
*Extract* the data from the exported file.
2.
*Create* a new OpenCms resource of the desired type.
3.
*Populate* the resource with the extracted data.
1) Extract the Data
This step depends heavily on the format of your export.
Let’s assume you’ve parsed the export into a |contentData| object, for
example:
contentData.get("title");
contentData.get("text");
contentData.get("teaser");
This could be a simple |HashMap<String, String>| or a dedicated Java
object with getters like |getTitle()|, |getText()|, etc.
2) Create or Update the Resource
String filename = [some logic to define the name of the new resource];
if (!cms.existsResource(filename)) {
// Create new resource
CmsResource res = cms.createResource(
filename,
OpenCms.getResourceManager().getResourceType("[RESOURCETYPE]")
);
CmsFile file = cms.readFile(res);
fillContent(cms, file, contentData);
} else {
// Update existing resource
CmsFile file = cms.readFile(filename);
cms.lockResource(file);
fillContent(cms, file, contentData);
}
*Explanation:*
*
|cms| → The current |CmsObject|
*
|[RESOURCETYPE]| → The name of the resource type you want to create
*
|fillContent()| → A helper method to insert the actual data (see below)
After this step, you have either a new empty resource or an existing one
ready to be updated.
3) Fill the Content
Here’s an example |fillContent()| method that writes your data into all
available locales:
CmsXmlContent content = CmsXmlContentFactory.unmarshal(cms, file);
List<Locale> locales = OpenCms.getLocaleManager().getAvailableLocales();
for (Locale locale : locales) {
if (!content.hasLocale(locale)) {
content.addLocale(cms, locale);
}
setValue(cms, content, XPATH_TITLE, contentData.get("title"), locale);
setValue(cms, content, XPATH_TEXT, contentData.get("text"), locale);
setValue(cms, content, XPATH_TEASER, contentData.get("teaser"),
locale);
}
// Marshal XML back into file and save
file.setContents(content.marshal());
cms.writeFile(file);
*Notes:*
*
|XPATH_TITLE|, |XPATH_TEXT|, and |XPATH_TEASER| refer to the XML
schema elements defined in your content type.
For example:
<xsd:element name="Title" type="OpenCmsString"/>
<xsd:element name="Text" type="OpenCmsString"/>
<xsd:element name="Teaser" type="OpenCmsString"/>
Corresponding XPaths would be |"Title"|, |"Text"|, and |"Teaser"|.
4) Set Value Helper Method
private void setValue(CmsObject cms, CmsXmlContent content, String
xpath, String value, Locale locale) {
int index = 0;
if (value == null) {
return;
}
if (!content.hasValue(xpath, locale, index)) {
content.addValue(cms, xpath, locale, index);
}
content.getValue(xpath, locale, index).setStringValue(cms, value);
}
This method ensures the XML element exists and safely inserts the
provided value.
Summary
*
*Never* write directly into the OpenCms database.
*
*Always* use the OpenCms API to create and populate resources.
*
Prepare your data export first (XML/JSON/CSV).
*
Use a simple Java importer to create or update content.
This approach is reliable, maintainable, and ensures that your data
stays consistent with OpenCms’s internal structure.
Kind regards,
Michael
Am 16.10.25 um 11:34 schrieb Atle Enersen via opencms-dev:
>
> Hello.
>
> We have some hundred (at least) articles in our old system that we
> wold like to insert into OpenCms automagically. We are not gonna take
> all, but that’s not OpenCms’ problem.
>
> We have a working structure on the old site, so filtering content is
> manageable. Old articles will be marked as ... old.
>
> Is there any documentation on this? Does som kind of database scheme
> exist for us external people? Are there any functions for this
> internally in OpenCms? I am aware that doing hack son the database
> would leave us «alone», no support.
>
> Thanks in advance :-)
>
> Vennlig hilsen, Atle Enersen
>
> --
>
> Tel.: +47 48 06 31 71
>
>
> _______________________________________________
> This mail is sent to you from the opencms-dev mailing list
> To change your list options, or to unsubscribe from the list, please visit
> https://lists.opencms.org/mailman/listinfo/opencms-dev
>
>
>
--
Michael Emmerich
-------------------
Alkacon Software GmbH & Co. KG - The OpenCms Experts
http://www.alkacon.com
http://www.opencms.org
--------------9jTrFn3rxbG1AIdBBnI6YReH
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello Atle,</p>
<p data-start="263" data-end="399">first of all — <strong
data-start="278" data-end="355">inserting data directly into the
OpenCms database is strongly discouraged</strong>. Doing so can
easily corrupt your system.</p>
<p data-start="401" data-end="631">The correct and safe way to
migrate or import data is to <strong data-start="458"
data-end="480">create an importer</strong> that uses the <strong
data-start="495" data-end="510">OpenCms API</strong> to generate
content based on the data from your existing system. Fortunately,
this is not as complicated as it sounds.</p>
<h3 data-start="638" data-end="686">Step 1: Export Data from Your
Current System</h3>
<p data-start="688" data-end="1003">Before you can import anything
into OpenCms, determine how you can <strong data-start="755"
data-end="765">export</strong> your articles from your current
system, and in what <strong data-start="818" data-end="828">format</strong>
(e.g., XML, JSON, CSV).<br data-start="852" data-end="855">
Each article should ideally be exported as a separate file — for
example, one XML file per article — or retrieved directly via an
interface (API).</p>
<p data-start="688" data-end="1003"><br>
</p>
<h3 data-start="1010" data-end="1045">Step 2: Import Process
Overview</h3>
<p data-start="1047" data-end="1117">For each article, your import
process will roughly follow these steps:</p>
<ol data-start="1119" data-end="1282">
<li data-start="1119" data-end="1168">
<p data-start="1122" data-end="1168"><strong data-start="1122"
data-end="1133">Extract</strong> the data from the exported
file.</p>
</li>
<li data-start="1169" data-end="1228">
<p data-start="1172" data-end="1228"><strong data-start="1172"
data-end="1182">Create</strong> a new OpenCms resource of
the desired type.</p>
</li>
<li data-start="1229" data-end="1282">
<p data-start="1232" data-end="1282"><strong data-start="1232"
data-end="1244">Populate</strong> the resource with the
extracted data.</p>
</li>
</ol>
<p><br>
</p>
<h4 data-start="1317" data-end="1341">1) Extract the Data</h4>
<p data-start="1343" data-end="1482">This step depends heavily on
the format of your export.<br data-start="1398" data-end="1401">
Let’s assume you’ve parsed the export into a <code
data-start="1446" data-end="1459">contentData</code> object, for
example:</p>
<p data-start="1343" data-end="1482">contentData.get("title");<br>
contentData.get("text");<br>
contentData.get("teaser");</p>
<p data-start="1343" data-end="1482">This could be a simple <code
data-start="1598" data-end="1623">HashMap<String, String></code>
or a dedicated Java object with getters like <code
data-start="1669" data-end="1681">getTitle()</code>, <code
data-start="1683" data-end="1694">getText()</code>, etc.</p>
<p data-start="1343" data-end="1482"><br>
</p>
<h4>2) Create or Update the Resource</h4>
<p>String filename = [some logic to define the name of the new
resource];<br>
<br>
if (!cms.existsResource(filename)) {<br>
// Create new resource<br>
CmsResource res = cms.createResource(<br>
filename,<br>
OpenCms.getResourceManager().getResourceType("[RESOURCETYPE]")<br>
);<br>
CmsFile file = cms.readFile(res);<br>
fillContent(cms, file, contentData);<br>
} else {<br>
// Update existing resource<br>
CmsFile file = cms.readFile(filename);<br>
cms.lockResource(file);<br>
fillContent(cms, file, contentData);<br>
}</p>
<p data-start="2267" data-end="2283"><strong data-start="2267"
data-end="2283">Explanation:</strong></p>
<ul data-start="2284" data-end="2467">
<li data-start="2284" data-end="2319">
<p data-start="2286" data-end="2319"><code data-start="2286"
data-end="2291">cms</code> → The current <code
data-start="2306" data-end="2317">CmsObject</code></p>
</li>
<li data-start="2320" data-end="2391">
<p data-start="2322" data-end="2391"><code data-start="2322"
data-end="2338">[RESOURCETYPE]</code> → The name of the
resource type you want to create</p>
</li>
<li data-start="2392" data-end="2467">
<p data-start="2394" data-end="2467"><code data-start="2394"
data-end="2409">fillContent()</code> → A helper method to
insert the actual data (see below)</p>
</li>
</ul>
<p data-start="2469" data-end="2564">After this step, you have
either a new empty resource or an existing one ready to be
updated.</p>
<p data-start="2469" data-end="2564"><br>
</p>
<h4 data-start="2571" data-end="2595">3) Fill the Content</h4>
<p data-start="2597" data-end="2687">Here’s an example <code
data-start="2615" data-end="2630">fillContent()</code> method
that writes your data into all available locales:</p>
<p data-start="2469" data-end="2564">CmsXmlContent content =
CmsXmlContentFactory.unmarshal(cms, file);<br>
List<Locale> locales =
OpenCms.getLocaleManager().getAvailableLocales();<br>
<br>
for (Locale locale : locales) {<br>
if (!content.hasLocale(locale)) {<br>
content.addLocale(cms, locale);<br>
}<br>
<br>
setValue(cms, content, XPATH_TITLE, contentData.get("title"),
locale);<br>
setValue(cms, content, XPATH_TEXT, contentData.get("text"),
locale);<br>
setValue(cms, content, XPATH_TEASER,
contentData.get("teaser"), locale);<br>
}<br>
<br>
// Marshal XML back into file and save<br>
file.setContents(content.marshal());<br>
cms.writeFile(file);</p>
<p data-start="3285" data-end="3295"><strong data-start="3285"
data-end="3295">Notes:</strong></p>
<ul data-start="3296" data-end="3662">
<li data-start="3296" data-end="3662">
<p data-start="3298" data-end="3425"><code data-start="3298"
data-end="3311">XPATH_TITLE</code>, <code data-start="3313"
data-end="3325">XPATH_TEXT</code>, and <code
data-start="3331" data-end="3345">XPATH_TEASER</code> refer
to the XML schema elements defined in your content type.<br
data-start="3408" data-end="3411">
For example:</p>
</li>
</ul>
<p><xsd:element name="Title" type="OpenCmsString"/><br>
<xsd:element name="Text" type="OpenCmsString"/><br>
<xsd:element name="Teaser" type="OpenCmsString"/></p>
<p data-start="688" data-end="1003">Corresponding XPaths would be <code
data-start="3626" data-end="3635">"Title"</code>, <code
data-start="3637" data-end="3645">"Text"</code>, and <code
data-start="3651" data-end="3661">"Teaser"</code>.</p>
<p data-start="401" data-end="631"><br>
</p>
<h4 data-start="401" data-end="631">4) Set Value Helper Method</h4>
<p>private void setValue(CmsObject cms, CmsXmlContent content,
String xpath, String value, Locale locale) {<br>
int index = 0;<br>
if (value == null) {<br>
return;<br>
}<br>
if (!content.hasValue(xpath, locale, index)) {<br>
content.addValue(cms, xpath, locale, index);<br>
}<br>
content.getValue(xpath, locale, index).setStringValue(cms,
value);<br>
}<br>
<br>
</p>
<p>This method ensures the XML element exists and safely inserts the
provided value.</p>
<p data-start="401" data-end="631"><br>
</p>
<h3 data-start="4200" data-end="4211">Summary</h3>
<ul data-start="4213" data-end="4448">
<li data-start="4213" data-end="4268">
<p data-start="4215" data-end="4268"><strong data-start="4215"
data-end="4224">Never</strong> write directly into the
OpenCms database.</p>
</li>
<li data-start="4269" data-end="4337">
<p data-start="4271" data-end="4337"><strong data-start="4271"
data-end="4281">Always</strong> use the OpenCms API to
create and populate resources.</p>
</li>
<li data-start="4338" data-end="4388">
<p data-start="4340" data-end="4388">Prepare your data export
first (XML/JSON/CSV).</p>
</li>
<li data-start="4389" data-end="4448">
<p data-start="4391" data-end="4448">Use a simple Java importer
to create or update content.</p>
</li>
</ul>
<p data-start="4450" data-end="4569">This approach is reliable,
maintainable, and ensures that your data stays consistent with
OpenCms’s internal structure.</p>
<p data-start="401" data-end="631"><br>
</p>
<p>Kind regards,</p>
<p>Michael</p>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Am 16.10.25 um 11:34 schrieb Atle
Enersen via opencms-dev:<br>
</div>
<blockquote type="cite"
cite="mid:VI1PR10MB2063E03FE0A196E4F0ADB350A9E9A@VI1PR10MB2063.EURPRD10.PROD.OUTLOOK.COM">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator"
content="Microsoft Word 15 (filtered medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;
mso-fareast-language:EN-US;}span.EpostStil17
{mso-style-type:personal-compose;
font-family:"Aptos",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
mso-fareast-language:EN-US;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal">Hello.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We have some hundred (at least) articles in
our old system that we wold like to insert into OpenCms
automagically. We are not gonna take all, but that’s not
OpenCms’ problem.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We have a working structure on the old
site, so filtering content is manageable. Old articles will be
marked as ... old.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Is there any documentation on this? Does
som kind of database scheme exist for us external people? Are
there any functions for this internally in OpenCms? I am aware
that doing hack son the database would leave us «alone», no
support.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks in advance :-)<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span
style="mso-ligatures:none;mso-fareast-language:NO-BOK">Vennlig
hilsen, Atle Enersen<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="mso-ligatures:none;mso-fareast-language:NO-BOK">-- <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="mso-ligatures:none">Tel.:
+47 48 06 31 71<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre wrap="" class="moz-quote-pre">_______________________________________________
This mail is sent to you from the opencms-dev mailing list
To change your list options, or to unsubscribe from the list, please visit
<a class="moz-txt-link-freetext" href="https://lists.opencms.org/mailman/listinfo/opencms-dev">https://lists.opencms.org/mailman/listinfo/opencms-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
Michael Emmerich
-------------------
Alkacon Software GmbH & Co. KG - The OpenCms Experts
<a class="moz-txt-link-freetext" href="http://www.alkacon.com">http://www.alkacon.com</a>
<a class="moz-txt-link-freetext" href="http://www.opencms.org">http://www.opencms.org</a>
</pre>
</body>
</html>
--------------9jTrFn3rxbG1AIdBBnI6YReH--
--===============7171036632105921868==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KVGhpcyBtYWls
IGlzIHNlbnQgdG8geW91IGZyb20gdGhlIG9wZW5jbXMtZGV2IG1haWxpbmcgbGlzdApUbyBjaGFu
Z2UgeW91ciBsaXN0IG9wdGlvbnMsIG9yIHRvIHVuc3Vic2NyaWJlIGZyb20gdGhlIGxpc3QsIHBs
ZWFzZSB2aXNpdApodHRwczovL2xpc3RzLm9wZW5jbXMub3JnL21haWxtYW4vbGlzdGluZm8vb3Bl
bmNtcy1kZXYKCgoK
--===============7171036632105921868==--