Solr is one of the highly reliable search engine which comes along with WebSphere Commerce Server. By default, Solr contain required configuration to support the search features used by WebSphere Commerce Server. But, during development, there can be some situation where we need to create an additional solr core and configuration to index additional details to Solr to improve the performance of certain functionalities.
Given below steps explains how to create a new Solr core in WebSphere Commerce Server.
Create a new Solr core
Step 1 - Navigate to C:\IBM\WCDE80\search\solr\home and edit solr.xml file. Create a new Solr core by adding the following lines (in bold) inside the <core> tag –
<cores>
…
…
<core instanceDir="MC_10001\en_US\CustomData\" name="CustomData_en_US"/>
</cores>
Step 2 – Create a folder structure to place the configuration and data files for indexing the Store Locator details
Navigate to C:\IBM\WCDE80\search\solr\home\MC_10001\en_US and create “CustomData” folder.
Now navigate to this folder and then create 2 sub folders:
1. conf – to hold the Solr configuration files
2. data – to store the indexes
Step 3 - Create configuration files to support the custom data
To speed up the configuration changes, let us copy the existing configuration files from C:\IBM\WCDE80\search\solr\home\MC_10001\en_US\CatalogEntry\conf to the newly created conf folder under CustomData.
a) Open schema.xml and navigate to the section starting with the text "Websphere Commerce text field naming convention". Below that, you can see the field definition created for CATENTRY information. Replace the field definition with the custom field details we need to index into Solr.
<field name="name" type="wc_text" indexed="true" stored="true" multiValued="false"/>
<field name="description" type="string" indexed="false" stored="true" multiValued="false"/>
<field name="brand" type="wc_text" indexed="true" stored="true" multiValued="true"/>
<field name="brand_ntk" type="wc_keywordTextLowerCase" indexed="true" stored="true" multiValued="true"/>
<copyField source="brand" dest="brand_ntk"/>
<!-- Field to use to determine and enforce document uniqueness.
Unless this field is marked with required="false", it will be a required field
-->
<uniqueKey>name</uniqueKey>
<!-- field for the QueryParser to use when an explicit fieldname is absent -->
<defaultSearchField>name</defaultSearchField>
b) update wc-data-config.xml file with the SQL queries and field definitions to load the data to Solr from database (if the source is present in database. It is also possible to load contents directly from files which I will explain in my next blog).
After the changes, final wc-data-config.xml will look similar to the content given below.
<!-- WebSphere Commerce Solr Data Import Handler configuration -->
<dataConfig>
<dataSource name="WC database"
type="com.ibm.commerce.solr.handler.SchemaJdbcDataSource"
jndiName="com.ibm.commerce.foundation.server.services.search.datasource"
readOnly="true"
autoCommit="true"
transactionIsolation="TRANSACTION_READ_COMMITTED"
holdability="CLOSE_CURSORS_AT_COMMIT"
fullyMaterializeLobData="true"
fullyMaterializeInputStreams="true"
progressiveStreaming="2"
progresssiveLocators="2"
batchSize="1000"
/>
<document name="CustomData">
<entity name="CustomInfo"
dataSource="WC database"
transformer="ClobTransformer, RegexTransformer, com.ibm.commerce.solr.handler.NameValuePairTransformer"
query="SELECT TEST.NAME, TEST.DESCRIPTION, TEST.BRAND FROM TEST"
>
<field column="NAME" name="name" />
<field column="DESCRIPTION" name="description" />
<field column="BRAND" name="brand" />
</entity>
</document>
</dataConfig>
Note: If the data to be indexed (eg. brand) is present in a different table and if "name" is the primary key of the second table, the following entry in wc-data-config.xml inside <entity> will help to include brand information in the same record.
<entity name="brand" query="SELECT brand FROM BRANDTEST where name = ${CustomInfo.NAME} ">
<field column="brand" name="BRAND"/>
</entity>
Step 4 - Restart the Solr server
Step 5 - Verify whether the core is created properly or not by hitting the following url
http://localhost:81/solr/CustomData_en_US/select?q=*.*
You should get a response XML with no data in it.
Registering the index in WebSphere Commerce Database:
To index the data from WCS DB to Solr, the following configuration entries need to be inserted in search configuration tables.
1) INSERT INTO srchconf (indextype, indexscope,languages,config,optcounter) values ('CustomData',10001,'-1,-1001,-1002','IndexScopeTag=2,SearchServerPort=81,SearchServerName=localhost,PreProcessConfigDirectory=C:\IBM\WCDE80\search\pre-processConfig\MC_10001\DB2\CustomData',27);
2) INSERT INTO srchconfext (srchconfext_id,indextype,indexscope,language_id,indexsubtype,config) VALUES(101,'CustomData','10001',-1,'Structured','SearchServerName=localhost,SearchServerPort=81');
3) INSERT INTO srchconfext (srchconfext_id,indextype,indexscope,language_id,indexsubtype,config) VALUES(102,'CustomData','10001',-1,'Unstructured','SearchServerName=localhost,SearchServerPort=81');
4) INSERT INTO srchconfext (srchconfext_id,indextype,indexscope,language_id,indexsubtype,config) VALUES(103,'CustomData','10001',-1,'WebContent','SearchServerName=localhost,SearchServerPort=81');
Importing Data to Solr:
The data can be imported to Solr by hitting the below URL in browser.
http://localhost:81/solr/CustomData_en_US/dataimport?command=full-import
By default, wc-data-config.xml will be referenced by data import command. If data import is being executed in other environments, then the name of the config file can be specified as a parameter. For example: http://localhost:81/solr/CustomData_en_US/dataimport?command=full-import&config=wc-data-config.dev.xml
You can check whether indexing is completed or not by using the below URL.
http://localhost:81/solr/CustomData_en_US/dataimport?command=status
Once it is completed, hit the below URL again and you can see the result with the details indexed.
http://localhost:81/solr/CustomData_en_US/select?q=*.*
No comments:
Post a Comment