IBM Cloud Pak for Integration Support Guide for App Connect L2

Created by Kodzo Nyamadi on Mon, 09/26/2022 - 02:23
Published URL:
https://supportcontent.ibm.com/support/pages/node/1089418
1089418

Product Documentation


Abstract

The information on this page is to be used by cp4i support L2.

Content

Internal Use Only

What is IBM Cloud Pak for Integration?

Every enterprise in today's markets must offer robust digital products and services, often requiring integration of complex capabilities and systems to deliver one coherent whole. IBM Cloud Pak for Integration (CP4I) offers a simplified solution to this integration challenge, allowing the enterprise to modernize its processes while positioning itself for future innovation. IBM Cloud Pak for Integration eases monitoring, maintenance and upgrades, helping the enterprise stay ahead of the innovation curve. IBM Cloud Pak for Integration brings together proven, best-in-class capabilities to deliver a streamlined forward-looking modern integration solution, available as a single high value purchase at lower cost. This new product unifies disparate tools into one solution that integrates both modern and traditional products.

Components of Cloud Pak for Integration

CP4I is a platform designed to offer multiple containerized products. Below is a list of components that are integrated in CP4I

  Note:

Keep in mind that it is not because you open a skill case against a team that the team has to work it. If the team determines that the issue being investigated is not related to their code, they can send the skill case back to CP4I L2. We encourage teams that are sending cases back to CP4I L2 to provide a rational detail of why they think it is not their issue to investigate. This is a recommendation; not an obligation, so when you find yourself in this situation, read the case description again. If you still don't have a clue, use collaboration slack channels or work with your team CP4I focal to determine what team should you engage to further assist you in resolving the issue or what team should you collaborate with.

Types of Issues Descriptions Affected Components Support Teams
CP4I Installation Client downloaded the CP4I install imaged and is installing CP4I on either OpenShift or MS Azure, AWS, Google Cloud, IBM Cloud as new installation or trying to install CP4I chart into existing icp environment or via the cloudctl command. Installation is not being completed due to potential errors. You may also run into situations where clients are doing one-click install on IBM Cloud or using automation script. ICP, OpenShift, MS Azure, AWS, Google Cloud, IBM Cloud

CP4I L2 owns parent case

For Common Services issues during install, CP4I L2 to engage Common Services via CRF and request parent transfer or skills case engagement.

Common Services CRF

ICP Installation Client downloaded the CP4I install image, and he configured the config.yaml file to install ICP only. It is also possible that he may want to install the cp4i platform navigator later via the ICP catalog to take advantage of its features. Possible errors:
  • Installer (playbook) is failing.
  • helm-api pod(s) is not running.
  • 502 Bad Gateway error when accessing the Catalog or heml release page.
  • Any other related error to the ICP dashboard including login issues.
ICP, OpenShift, MS Azure, AWS, Google Cloud, IBM Cloud

CP4I L2 to transfer parent case to Common Services.

Common Services CRF

Product Specific Charts Installation from ICP Catalog It is also possible to install products charts (dashboards) from the ICP catalog including the CP4I Navigator. For example, a client might want to install ACE or MQ dashboard after installing CP4I Navigator. In this case they can do it either from the ICP catalog or from the CP4I Navigator. In case they decide to install a product specific chart from the ICP catalog and experience issues during the process, Common Services team should be engaged. ICP, OpenShift, MS Azure, AWS, Google Cloud, IBM Cloud

CP4I L2 to transfer parent case to product specific team mission.

For Common Services issues during install, CP4I L2 to engage Common Services via CRF and request parent transfer or skills case engagement.

Common Services CRF

CP4I Navigator CP4I install has been completed successfully. However, client is experiencing issue with the dashboard itself. These type of issues might includes:
  • Dashboard not displaying in browser.
  • Getting certificate error.
  • Dashboard pods are not running.
CP4I

CP4I L2 works parent case

CP4I L2 to engage CP4I L3 via collaboration case in Salesforce if cannot determine root cause.

Issues with a Product Specific Dashboard on CP4I Navigator Client has installed CP4I and was able to access its dashboard. From here, there are several things clients might want to do and that can go wrong. Common issues might be:
  • Create (also called deploy or install) a product specific instance or dashboard.
  • Unable to access product specific instance dashboard when launching from CP4I Navigator.
  • Any other issues with product specific on CP4I Navigator.
ICP, CP4I, Product Specific

CP4I L2 to transfer parent case to product specific team mission.

If Product specific L2 cannot resolve the issue and need further assistance, they should work with their CP4I focals to engage CP4I L3 or Common Services depending on the nature of the issue.

Issues on Product Specific Dashboard Here, we assume clients has installed CP4I, deploy or instance product specific dashboard or chart. They are also able to access the product specific chart from CP4I Navigator or other means. On the product specific dashboard, clients can face manay issues.
  • Cannot create Intergration Servers or queue managers.
  • Failed to deploy bar files.
  • Anything that falls under product specific usage falls under this category.
  • If the issue is related to ACE then follow the support guide to collect mustgather and handle the case as you normally would for standard IIB and ACE cases.
  • For any other product issue, you should transfer the parent case to that team for investigation.
ICP, Product Specific

CP4I L2 to transfer parent case to product specific team mission.

Issue a Product Specific Run-Time Client has installed CP4I and was able to access its dashboard. From here, there are several things clients might want to do and that can go wrong. Common issues might be:
  • Unable to deploy application.
  • Anything related to product usability.
  • Product specific pods is not running.
Product Specific

CP4I L2 to transfer parent case to product specific team mission.

Issues with ACE Pods Management Client has installed CP4I and was able to access its dashboard. From here, there are several things clients might want to do and that can go wrong. Common issues might be:
  • ACE pod is missing or not running.
  • ACE pods is in pending status or cannot be created.
  • Any other similar issues.
ICP, CP4I, ACE

CP4I L2 works parent case

For Common Services issues during install, CP4I L2 to engage Common Services via CRF and request parent transfer or skills case engagement.

Common Services CRF

Issues with Logging Components Client has installed CP4I and was able to access its dashboard. However, he is having issues logging components sucha as:
  • Kibana pod is not running.
  • Cannot access the Kibana dashboard.
  • Any other similiar issues.
ICP

For Common Services issues, CP4I L2 to engage Common Services via CRF and request parent transfer or skills case engagement.

Common Services CRF

Issues with Operational Dashboard (OD)

Operational Dashboard is an independent component in cloud pak for integration that provides tracing capabilities. Common issues with this component might be:

  • OD install is failing.
  • OD registration pod is not running.
  • Stats are not visible on OD.
  • Any other issue related to OD.
ICP, CP4I, OD

CP4I L2 works parent case

CP4I L2 to engage Montier Servic Desk via Montier Service Portal

Montier emailID:
ibm-support@mon-tier.com

CP4I INSTALLATION

Use this MustGather if a client is doing cp4i install following the steps in CP4I Knowledge Center

  1. Re-run the CP4I installer in verbose mode by appending this command to the installer command, and provide the output log: -vvv | tee ~/cp4i_install.log.

CP4I INSTALLATION WITH ONE-CLICK OR AUTOMATION SCRIPT

If a client is doing CP4I install on IBM Cloud using one-click procedure or the automation script,

  1. What is the OCP version: ?
  2. We'll also need to collect some information from cluster summary. (On ibm cloud Kubernetes -> clusters -> cluster name)
  3.  The 1-click install logs from the web-ui
  4. Run the below OC Commands after logging in (oc login) to the cluster.
    oc get pods -n kube-system
    oc get pods -n
    oc get routes --all-namespaces
    oc describe route navigator -n integration
    oc describe route icp-console -n kube-system
    oc describe route icp-proxy -n kube-system
    oc get events --all-namespaces

CP4I Navigator

  1. CP4I Version: cloudctl version.
  2. Platform Options: OpenShift version on [Redhat, MS Azure, AWS, Google Cloud, IBM Cloud, AWS, Google, IBM Cloud, Centos, Ubuntu].
  3. Output all pods in your namespace to text file: oc get pods -n <namespace>.
  4. Get the pod(s) log(s) in text file: oc logs -f <pod name> -c <container name> -n <namespace>. It is possible to have more than one pod; Better to get the logs for all if logs contents are not the same.
  5. Describe the pod(s) and redirect the output to a text file: oc describe pod <pod name> -n <namespace>. It is possible to have more than one pod; Better to describe all if they provide different outputs.
  6. Output all nodes in your namespace to atext file: oc get nodes -n <namespace>.
  7. Output of the ingress to text file: oc get ingress -n <namespace>.
  8. Output of the routes to text file: oc get routes.
  9. Output of the jobs to text file: oc get jobs -n <namespace>.
  10. Output of the pods in the kube-system namespace to text file: oc get pods -n kube-system.
  11. Output of the daemonsets in the kube-system to text file: oc get daemonsets -n kube-system.
  12. Get the tiller pod log in text file: oc logs <tiller pod name> -n <namespace>.

PRODUCT SPECIFIC DASHBOARD

If client is having any type of issue with a product specific dashboard, please transfer the parent case to that product specific team. In addition, if the issue is related to the run-time of a product specific, you would also transfer the parent case to that product team.

In case, it happens that the product specific is ACE, the CP4I team will take ownership of the parent case and work it. Please look at the ACE DASHBOARD section for the MustGather. If you need to engage L3 then you must engage ACE L3 via a skill case. Please review ACE_L2_L3 case template before opening the skill case.

ACE DASHBOARD

  1. https://docs.openshift.com/container-platform/4.6/welcome/index.html

ACE RUN-TIME: PODS MANAGEMENT

  1. CP4I Version: cloudctl version.
  2. Platform Options: OpenShift version on [Redhat, MS Azure, AWS, Google Cloud, IBM Cloud, AWS, Google, IBM Cloud, Centos, Ubuntu].
  3. Output all pods in your namespace to text file: oc get pods -n <namespace>.
  4. Get the pod(s) log(s) in text file: oc logs -f <pod name> -c <container name> -n <namespace>. It is possible to have more than one pod; Better to get the logs for all if logs contents are not the same.
  5. Describe the replicaset and redirect the output to a text file: kubectl describe rs/<replicaset name>.
  6. For ACE-specific logging, please refer to the ACE FAQ under the section Troubleshooting and Serviceability

ACE RUN-TIME: PODS CONTENTS

Clients might have issues with bar file deployment, execution of mqsi commands, accessing webui, or issue related to their project interchange (PI).

  1. CP4I Version: cloudctl version.
  2. Platform Options: OpenShift version on [Redhat, MS Azure, AWS, Google Cloud, IBM Cloud, AWS, Google, IBM Cloud, Centos, Ubuntu].
  3. Output all pods in your namespace to text file: oc get pods -n <namespace>.
  4. Get the pod(s) log(s) in text file: oc logs -f <pod name> -c <container name> -n <namespace>. It is possible to have more than one pod; Better to get the logs for all if logs contents are not the same.
  5. For ACE-specific logging, please refer to the ACE FAQ under the section Troubleshooting and Serviceability

Common Services & RHOCP

Clients are experiencing issues with one or more Common Services / RHOCP components packaged with CP4I

export MGDIR=cp-ibmcommonservices-$(date -Ihours)
export LOGLIMIT="--tail=10000"
mkdir -p $MGDIR

oc get nodes > $MGDIR/node-list.txt
oc describe nodes > $MGDIR/node-describe.txt
oc get namespaces > $MGDIR/namespaces.txt
oc get clusteroperators > $MGDIR/cluster-operators.txt
oc get pods --all-namespaces -o wide > $MGDIR/all-pods-list.txt
oc adm top nodes > $MGDIR/top-nodes.txt
oc adm top pods --all-namespaces > $MGDIR/top-pods.txt
oc get cm ibmcloud-cluster-info -n kube-public -o yaml > $MGDIR/ibmcloud-cluster-info.txt

for NS in ibm-common-services kube-system; do
  export NS=$NS; mkdir $MGDIR/$NS; echo gathering info from namespace $NS
  oc get all,secrets,cm,ing,events -n $NS -o wide &> $MGDIR/$NS/all-list.txt
  oc get pods -n $NS | awk 'NR>1{print "oc -n $NS describe pod "$1" > $MGDIR/$NS/"$1"-describe.txt && echo described "$1}' | bash
  oc get pods -n $NS -o go-template='{{range $i := .items}}{{range $c := $i.spec.containers}}{{println $i.metadata.name $c.name}}{{end}}{{end}}' > $MGDIR/$NS/container-list.txt
  awk '{print "oc -n $NS logs "$1" -c "$2" $LOGLIMIT -p > $MGDIR/$NS/"$1"_"$2"_previous.log && echo gathered previous logs of "$1"_"$2}' $MGDIR/$NS/container-list.txt | bash
  awk '{print "oc -n $NS logs "$1" -c "$2" $LOGLIMIT > $MGDIR/$NS/"$1"_"$2".log && echo gathered logs of "$1"_"$2}' $MGDIR/$NS/container-list.txt | bash
done

tar czf $MGDIR.tgz $MGDIR/
echo "DONE"
echo "--- Please send "$MGDIR.tgz" to Common Services Support Team ---"

Collaboration Case for CP4I Development Template

  • DESCRIPTION OF THE ISSUE
    • Provide as much description as possible of the problem.
  • ENVIRONMENT DETAILS
    • Platforms: [RedHat OpenShift, MS Azure, AWS, Google Cloud, IBM Cloud]
    • Versions: [CP4I version], [Platform Version]
    • CSP Case Number: [Parent Case Number]
    • CSP Case Severity: [Case Priority Number]
  • MUST GATHERS / DOCUMENTATION PROVIDED
    • Provide ecurep path to the documentation to be analyzed
  • IS THE PROBLEM RECREATABLE
    • Can the problem be easily recreated by either us or the customer outside of production?
  • ACTION PLAN
    • What is the suggested next action for CP4I L3 to perform
  • URGENCY
    • Production or Test/Development environment:
    • Customer deadlines:
    • Is this a critsit?

Skill Case for Common Services

  • PROBLEM SUMMARY
    • Short description of the problem
  • ENVIRONMENT DETAILS
    • Platforms: [RedHat OpenShift version, MS Azure, AWS, Google Cloud, IBM Cloud]
    • Versions: [CP4I version], [Platform Version]
  • MUST GATHERS/ DOCUMENTATION PROVIDED
    • Provide ecurep path to the documentation to be analyzed
  • IS THE PROBLEM RECREATABLE
    • Can the problem be easily recreated by either us or the customer outside of production?
  • ACTION PLAN
    • What is the suggested next action for Common Services to perform
  • URGENCY
    • Production or Test/Development environment:
    • Customer deadlines:
    • Is this a critsit?

Note:To find which product name, support mission, support skill to use when creating skill cases or transferring parent cases to Common Services team, please check the Product CRF & Escalations tab on this page

Skill Case for ACE L3 Template

Please follow the same template procedure as for ACE on premise.

Engaging Operations Dashbord Development - Montier

Please note: OD Dev do not work on Fridays and do work on Sundays.
Montier Email ID: 
ibm-support@mon-tier.com

  • Case Subject template
  • [Client Name][CSP Case Number][Case Subject Details]
  • PROBLEM SUMMARY
    • Short description of the problem
  • ENVIRONMENT DETAILS
    • Platforms: [RedHat OpenShift version, MS Azure, AWS, Google Cloud, IBM Cloud]
    • Versions: [OpenShift version],[CP4I version], [OD image version]
  • MUST GATHERS/ DOCUMENTATION PROVIDED
    • Provide ecurep path to the documentation to be analyzed
  • IS THE PROBLEM RECREATABLE
    • Can the problem be easily recreated by either us or the customer outside of production?
  • URGENCY
    • Production or Test/Development environment:
    • Customer deadlines:
    • Is this a critsit?

CP4I 2020.4.2 / OpenShift 4.6 FYRE LAB DETAILS

DATE VIDEO URL SPEAKER TOPICS
2019-10-15 Session 01 Hasan Rizvi CP4I Installation Overview:Supported Platforms and Type of installations
2019-10-22 Session 02 Hasan Rizvi CP4I Navigator overview, ACE instance creation
2019-10-30 Session 03 Hasan Rizvi Creation of ACE dashboard, Bar file deployment
2019-11-01 Session 04 KB Nyamadi, Matthew Segall ACE Instance, dashboard, server creation. Run of mqsiprofile and mqsi commands
2019-11-04 Session 05 KB Nyamadi CP4I Installation from Scratch
2019-11-05 Montier Session 01 Amit Munwes Operational Dashaboard overview and installation walk through
2019-11-11 Montier - Resolve Issue with OD Amit Munwes Error msg: Readiness probe failed: dial tcp 127.0.0.1:7600: connect: connection refused. Error: secrets "icp4i-od-store-cred" not found
2019-11-13 Montier Session 02 Amit Munwes Operation Dashboard installation, Troubleshooting: Error msg: Readiness probe failed: dial tcp 127.0.0.1:7600: connect: connection refused. Error: secrets "icp4i-od-store-cred" not found
2019-12-18 Session 06 Rex Reynolds CP4I 2019.4.1 Installation on Openshift 4.2 Demo
01-07-2020 Session 07 James Hewitt and Dan Pink 1-click public cloud / entitled registry & global catalog based install (on ROKS)
06-02-2020 ACE L3 Session 01

Password: tNbgGBG7

Amar Shah ACE Container overview and Dynamic Configuration

Slide Deck

06-09-2020 ACE L3 Session 02

Password: pEM3upY3

Amar Shah Connecting ACE Message Flow via ODBC
06-16-2020 ACE L3 Session 03

Password: TpMinqJ7

Amar Shah Connecting ACE Message Flow via MQ + Tracing capabilities

Slide Deck

06-16-2020 ACE Container w/ AVP

Password: AyNPKVg8

Paul Faulkner Overview of ACE on OpenShift (no Cp4I)
06-19-2020 ACE Container w/ AVP

Password: uR36TpbZ

Paul Faulkner Docker 101

Docker Cheat Sheet

06-23-2020 ACE L3 Session 04

Password: gVKeEGv3

Amar Shah KeyStore/Truststore in ACE Container

Slide Deck

06-30-2020 ACE L3 Session 05

Password: 2Mi5jPN2

Amar Shah ACE Container & Kafka

Slide Deck

07-08-2020 CP4I 2020.2 Operators skillshare w/ CP4IDev James Hewitt Operators and Cp4I 2020.2 release

Bonus Tech Sales knowledge transfer

PRODUCT CHANNELS
CP4I Core Platform and Asset Repository
  • #icp4i-tech - For internal tech sales support
  • #icp4i-support-escalations - Case escalations and collaboartion with the Public Cloud - ACS Team
  • #cp4i-l2-l3-collaboration - Collaboration with the CP4IDev Team
App Connect Enterprise
  • #appcon-acecc - Discussion of ACE on Certified Containers
  • #appcon-ace-questions - General questions regarding ACE
  • #appcon-aceoc - Discussion of ACE on Cloud
API Connect & DataPower
  • #engage_apic_dp_l2 - Collaboration with the APIC and DataPower Support Team
  • #apiconnect-techsales - Technical channel for client Advocates
  • #2018-apic-support - Technical channel for APIC on Cloud
Aspera
  • #aspera-support - Collaboration Channel for Aspera
MQ & Event Streams
  • #msg-es-l2-general - General Support Channel for Event Streams
  • #eventstreams-onprem - Channel for internal tech sales support
  • #msg-mq-l2-general - General channel for MQ Container issues
  • #mq-container-users - Collaboration channel for users of MQ Certified containers
Operations Dashboard - Montier
  • #icp4-trace - for Development issues
  • #montier-od-l2-collaboration - Collaboration channel for OD Dev and Support
  • Montier EmailID : ibm-support@mon-tier.com
Common Services
  • #icp-support-content - L2 Support Collaboration channel
  • #cloudpaks-common_svcs - General channel for Common Services questions
  • #private-cloud - general channel for IBM Cloud Private
Public Cloud - ACS Fabric To discuss with the ACS-Fabric team, @acs-fabric-team in your Cloud Pak slack channel (#icp4i-support-escalations), this will mention everyone on the ACS-Fabric team
RedHat OpenShift
  • #cloudpak-rhocp-support - Collaboration channel for RedHat issues
  • #rhm-support-collaboration - RedHat MarketPlace collaboration channel

eCustomerCare (eCare) is a single point of contact for internal and external customer questions related to the IBM DSW Web applications, Passport Advantage Online, Software Quote and Order (SQO), and Partner Guided Selling (PGS). Questions regarding access, registration, software download, media access, reporting functionality, and the navigation of the DSW web sites should be directed to eCare. eCare does not provide product-specific technical support (Product Support).

Please refer to the eCare CRF for CP4I entitlement concerns

Please refer to the RedHat Entitlement and General FAQ for general entitlement concerns and initial email communications sent to our clients.

OC2 has internal process for engaging CP4I L3. For more information on this process, please refer to the support wiki. Please, note that this wiki contains private data and is only access by OC2 support.

CP4I L3 CRF

Common Services CRF

DataPower Gateway and API Connect CRF

IBM MQ, IBM MQ Appliance and IBM Event Streams CRF

Hybrid Cloud Integration Duty ManagerCRF

DataPower Gateway and API Connect CRF

Public Cloud - Cloud Pak CRF

App Connect Designer CRF

Aspera CRF

RedHat Marketplace (RHMP) Support Process and CRF

Document Location

Worldwide

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS8QTD","label":"IBM Cloud Pak for Integration"},"Component":"","Platform":[{"code":"PF040","label":"RedHat OpenShift"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
26 September 2022

UID

ibm11089418