Home > Software design >  Cannot see any data in the Marklogic view
Cannot see any data in the Marklogic view

Time：05-11

I have created a template to create a view in the Marklogic database. The view is created successfully. And using mlcp import to ingest data from the XML file into the view. There is no error in response to the mlcp script which I am executing in my terminal windows. But I cannot see any data inserted into the view.

Please can you help in pin pointing the error please.

This is my template -

    import module namespace tde = "http://marklogic.com/xdmp/tde" at "/MarkLogic/tde.xqy";

if (xdmp:database-name(xdmp:database()) = "Documents")  then        
    let $shipment-CBE:=
      <template xmlns="http://marklogic.com/xdmp/tde">
 <path-namespaces>
  <path-namespace>
    <prefix>ns00</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/PurchaseOrderHeader</namespace-uri>
  </path-namespace>
  <path-namespace>
    <prefix>ns0</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/ConsignmentHeader</namespace-uri>
  </path-namespace>
<path-namespace>
    <prefix>ns1</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/PurchaseOrderDrop</namespace-uri>
  </path-namespace>
  <path-namespace>
    <prefix>ns2</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/ItemDetail</namespace-uri>
  </path-namespace>
  <path-namespace>
    <prefix>ns7</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/PurchaseOrderDelivery</namespace-uri>
  </path-namespace>
 <path-namespace>
    <prefix>ns9</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/BaseType</namespace-uri>
  </path-namespace>
<path-namespace>
    <prefix>ns10</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/ASN</namespace-uri>
  </path-namespace>
<path-namespace>
    <prefix>ns11</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/TransportationUnitHeader</namespace-uri>
  </path-namespace>
<path-namespace>
    <prefix>ns13</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/ContainmentHeader</namespace-uri>
  </path-namespace>
<path-namespace>
    <prefix>ns14</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/CargoBookingHeader</namespace-uri>
  </path-namespace>
<path-namespace>
    <prefix>ns15</prefix>
    <namespace-uri>http://www.globaltrademanagement_export.com/Consignment</namespace-uri>
  </path-namespace>
</path-namespaces>
  <context>/Shipment</context>
       <collections>
         <collections-and>
           <collection>PowerBI</collection>
           <collection>Shipment</collection>
         </collections-and>
       </collections>
       <rows>
         <row>
           <schema-name>Shipment</schema-name>
           <view-name>Test2</view-name>
            <columns>
             <column>
               <name>StatusID</name>
               <scalar-type>string</scalar-type>
               <val>ns9:DocumentHeader/ns9:Version/ns9:CurrentVersion/ns9:DocStatus/ns9:StatusID</val>
             </column>
              <column>
               <name>HouseBillRef</name>
               <scalar-type>string</scalar-type>
               <val>ns9:DocumentHeader/ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns0:ConsignmentHeader/ns0:HouseBillRef</val>
             </column>
              <column>
               <name>HouseBillDate</name>
               <scalar-type>string</scalar-type>
               <val>ns9:DocumentHeader/ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns0:ConsignmentHeader/ns0:HouseBillDate</val>
             </column>
              <column>
               <name>MasterBill</name>
               <scalar-type>string</scalar-type>
               <val>ns9:DocumentHeader/ns10:ASN/ns10:Schedule/ns11:TransportationUnitHeader/ns11:PrimarytUnit/ns11:TransportUnit/ns11:BillOfLading</val>
             </column>
              <column>
               <name>VesselNamePrfSch</name>
               <scalar-type>string</scalar-type>
               <val>ns9:DocumentHeader/ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns15:Bookings/ns14:CargoBookingHeader/ns14:DefaultSchedule/ns11:TransportationUnitHeader/ns11:PrimarytUnit/ns11:TransportUnit/ns11:TransUnitName</val>
             </column>
            <column>
               <name>POL_ETD_Prf_Sch</name>
               <scalar-type>dateTime</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns15:Bookings/ns14:CargoBookingHeader/ns14:DefaultSchedule/ns5:TransportationUnitHeader/ns5:PrimarytUnit/ns5:TransportUnit/ns5:PortOfLoading/ns9:ScheduledDepartureDate</val>
             </column>
             <column>
               <name>POD_ETA_Prf_Sch</name>
               <scalar-type>dateTime</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns15:Bookings/ns14:CargoBookingHeader/ns14:DefaultSchedule/ns5:TransportationUnitHeader/ns5:PrimarytUnit/ns5:TransportUnit/ns5:PortOfDischarge/ns9:ScheduledArrivalDate</val>
             </column>
              <column>
               <name>Trans_Mode</name>
               <scalar-type>string</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:TransMode</val>
             </column>
             <column>
               <name>Shipping_Terms</name>
               <scalar-type>string</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:ShipmentTerms</val>
             </column>
              <column>
               <name>LoadType</name>
               <scalar-type>string</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:SeaLoadType</val>
             </column>
             <column>
               <name>CargoDescription</name>
               <scalar-type>string</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:CargoDescription/ns9:CargoDescr</val>
             </column>
             <column>
               <name>PackType</name>
               <scalar-type>string</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:CargoDescription/ns9:Packaging/ns9:Package/ns9:PackageType</val>
             </column>
             <column>
               <name>PackQty</name>
               <scalar-type>unsignedLong</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:CargoDescription/ns9:Packaging/ns9:Package/ns9:PackageQty</val>
             </column>
              <column>
               <name>CBM</name>
               <scalar-type>decimal</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:CargoDescription/ns9:Packaging/ns9:Package/ns9:WeightDims/ns9:M3</val>
             </column>
             <column>
               <name>Net_Weight</name>
               <scalar-type>decimal</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:CargoDescription/ns9:Packaging/ns9:Package/ns9:WeightDims/ns9:NetWeight</val>
             </column>
             <column>
               <name>Gross_Weight</name>
               <scalar-type>decimal</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:CargoDescription/ns9:Packaging/ns9:Package/ns9:WeightDims/ns9:GrossWeight</val>
             </column>
             <column>
               <name>Litres</name>
               <scalar-type>decimal</scalar-type>
               <val>ns10:ASN/ns10:TrackedItem/ns15:Consignment/ns8:ConsignmentHeader/ns8:CargoSummary/ns8:CargoDescription/ns9:Packaging/ns9:Package/ns9:WeightDims/ns9:Litres</val>
             </column>
            </columns>
           </row>
           </rows>
         </template>
    return ( 
      tde:template-insert("/powerbi/Test2.xml",$shipment-CBE),
      "shipmentCBE.xml OK"
    )
else ("Please select the 'Documents' database.")

Response after running the mlcp script -

C:\Users\admin-rp\Documents\Marklogic\ML_With_PowerBI\bi-tools-master\bi-tools-master\power-bi\marklogic_powerbi_tutorial\config>mlcp.bat -options_file CBETestFileInsert.txt
22/05/10 14:39:59 DEBUG contentpump.ContentPump: Command: IMPORT
22/05/10 14:39:59 DEBUG contentpump.ContentPump: Arguments: -input_file_path ../data/CBE00030166N.xml -username admin -password ... -input_file_type documents -document_type xml -host owc-db01.owc.com -port 8000 -output_collections PowerBI,Shipment
22/05/10 14:39:59 DEBUG contentpump.ContentPump: Running in: localmode
22/05/10 14:39:59 INFO contentpump.LocalJobRunner: Content type: XML
22/05/10 14:39:59 INFO contentpump.ContentPump: Job name: local_240243545_1
22/05/10 14:39:59 INFO contentpump.FileAndDirectoryInputFormat: Total input paths to process : 1
22/05/10 14:39:59 DEBUG contentpump.CombineDocumentInputFormat: Total # of splits: 1
22/05/10 14:39:59 DEBUG contentpump.CombineDocumentInputFormat: Total # of combined splits: 1
22/05/10 14:39:59 DEBUG mapreduce.ContentOutputFormat: init query:
fn:exists(xdmp:get-request-header('x-forwarded-for'));
let $xdbcHeaderf := fn:function-lookup(xs:QName('xdmp:get-xdbc-request-header'),1)
return if (exists($xdbcHeaderf)) then fn:exists($xdbcHeaderf('x-forwarded-for')) else false();
import module namespace hadoop = "http://marklogic.com/xdmp/hadoop" at "/MarkLogic/hadoop.xqy";
xdmp:host-name(xdmp:host()),
let $versionf :=   fn:function-lookup(xs:QName('xdmp:effective-version'),0)
return if (exists($versionf)) then $versionf() else 0,
let $repf :=   fn:function-lookup(xs:QName('hadoop:get-forest-replica-hosts'),2)
return exists($repf),let $segRepf := fn:function-lookup(xs:QName('hadoop:get-forest-replica-hosts-with-segment'),2)
return exists($segRepf),let $f :=   fn:function-lookup(xs:QName('hadoop:get-assignment-policy'),0)
return if (exists($f)) then $f() else ()
22/05/10 14:39:59 DEBUG mapreduce.ContentOutputFormat: HTTP compliant mode disabled since x-forwarded-for doesn't exist
22/05/10 14:39:59 DEBUG contentpump.ThreadManager: Initial thread pool size: 32
22/05/10 14:39:59 DEBUG contentpump.ThreadManager: Thread pool is fixed and will not auto-scale.
22/05/10 14:39:59 DEBUG contentpump.ThreadManager: Running with MultithreadedMapper. Initial thread count for split #0: 32
22/05/10 14:40:00 INFO contentpump.LocalJobRunner:  completed 100%
22/05/10 14:40:00 INFO contentpump.LocalJobRunner: com.marklogic.mapreduce.MarkLogicCounter:
22/05/10 14:40:00 INFO contentpump.LocalJobRunner: INPUT_RECORDS: 1
22/05/10 14:40:00 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS: 1
22/05/10 14:40:00 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS_COMMITTED: 1
22/05/10 14:40:00 INFO contentpump.LocalJobRunner: OUTPUT_RECORDS_FAILED: 0
22/05/10 14:40:00 INFO contentpump.LocalJobRunner: Total execution time: 1 sec

And this is the mlcp script -

IMPORT
-input_file_path
../data/CBE00030166N.xml
-username
admin
-password
*****
-input_file_type 
documents 
-document_type 
xml
-host
owc-db01.owc.com
-port
8000
-output_collections
PowerBI,Shipment

CodePudding user response：

You are inserting the documents as the admin user. Are you querying and attempting to read them also as the admin user?

I don't see any explicit permissions declared for that TDE.

Double check what permissions are set for the view, and what the user you are attempting to query with have. Do they have the tde-view role?

Security on TDE Documents

Operations on template documents are controlled by:

The http://marklogic.com/xdmp/tde collection, which is a protected collection that contains TDE template documents.

The tde-admin role, which is required to access the TDE protected collection.

The tde-view role, which is required to view documents in the TDE protected collection. Access to views can be further restricted by setting additional permissions on the template documents that define the views. Since the same view can be declared in multiple templates loaded with different permissions, the access to views must be controlled at the column level as follows:

Column level read permissions are implicit by default and are derived from the read permissions set on the template documents. Permissions can also be explicitly set on a column using the permissions element. Permissions on a column are not required to be identical and are ORed together. A user with a role that has at least one of the read permissions set on a column will be able to see the column.

If a user does not have permissions on any of the view's columns, the view itself is not visible.

CodePudding user response：

Solved the issue:

There was a namespace which when omitted allows me to see the view. xmlns="http://www.globaltrademanagement_export.com/Shipment

This was present in the input xml file though. But surprisingly there was no error while running the mlcp import script.

CodePudding user response：

The view may have no reason to raise an error. Perhaps it is still a valid view and the context just matched nothing.. so no error..

I suspect that just nothing matched since you have your template had the context set to "/Shipment" and not "/Some-namesace:Shipment"

To help troubleshoot these items, I would suggest using :tde:node-data-extraxt()

Page link：https//www.codepudding.com/Softwaredesign/402452.html

Prev:XSD assuring that each enumeration value can only occur once as attribute
Next:Snowflake: Extracting multiple XML tags of same type "together"
  • Related

About Us:  Contact Us      Terms of Service       Privacy Policy

Copyright © 2010-2023，Powered By CodePudding