Showing posts with label Biztalk. Show all posts
Showing posts with label Biztalk. Show all posts

Tuesday, August 21, 2007

Where are the IIS and HTTPERR log files?

You can often diagnose HTTP adapter issues by using the IIS and HTTPERR log files supplied by the IIS server. By default, the IIS log files can be found in %WinDir%\system32\LogFiles\W3SVC1\ and the HTTPERR log files can be found in %WinDir%\system32\LogFiles\HTTPERR.

Note

The HTTPERR log file can only be found on a Windows Server 2003-based computer.

Source: Microsoft

HTTP Adapter

The HTTP adapter is used to exchange information between Microsoft BizTalk Server and an application by means of the HTTP protocol. The HTTP adapter consists of two adapters—a receive adapter and a send adapter. The HTTP receive adapter is a Microsoft Internet Information Services (IIS) Internet Server Application Programming Interface (ISAPI) extension that the IIS process hosts, and it controls the receive locations that use the HTTP adapter. The HTTP send adapter controls the send ports that use the HTTP adapter.

Source: Microsoft

Base EDI Adapter

The Microsoft Base EDI adapter provides comprehensive electronic data interchange (EDI) messaging functionality to complement the XML messaging capabilities of BizTalk Server 2006.

Source: Microsoft

File Adapter

The File adapter is in charge of reading and writing messages from files to and from the BizTalk Server MessageBox database.

Source: Microsoft

FTP Adapter

The FTP adapter exchanges data between an FTP server and Microsoft BizTalk Server, and allows for the integration of vital data stored on a variety of platforms with line-of-business applications.

Source: Microsoft

FTP receive adapter fails to publish message if receive pipeline processing time exceeds server time-out

Problem

A message received via the FTP adapter is not published to the MessageBox database. You may see errors similar to the following in the Application log for the BizTalk Server computer:

Event Type:Error

Event Source:BizTalk Server 2006

Event Category:BizTalk Server 2006

Event ID:5719

Date:6/30/2006

Time:12:08:55 PM

User:N/A

Computer:BIZTALKSERVER

Description:

There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.PassThruReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "Pipeline " Receive Port: "ReceivePort1" URI: "ftp://FTPSERVER:21/*.txt" Reason: Unable to receive the file "file.txt " from the FTP server.

and

Event Type:Warning

Event Source:BizTalk Server 2006

Event Category:BizTalk Server 2006

Event ID:5740

Date:6/30/2006

Time:12:08:56 PM

User:N/A

Computer:BIZTALKSERVER

Description:

The adapter "FTP" raised an error message. Details "Unable to receive the file "file.txt " from the FTP server. ".

Cause

The time taken to complete pipeline processing exceeds the FTP connection time-out.

Resolution

Use one of the following methods to mitigate this problem:

· Increase the connection time-out value. Set the idle time-out value on the FTP server to be at least the amount of the time it takes to process the file.

· Use the Temporary Folder feature on the receive location. In this case, the FTP adapter copies the file to the temporary folder (typically the local disk). It typically takes less time to copy the file to the local disk than to run the message through the pipeline and persist it to the MessageBox database, which effectively reduces the idle time.

Source: Microsoft

Configuring an FTP Adapter to Work with Legacy Hosts

This topic addresses what you need to know to facilitate communication between the FTP adapter and a mainframe computer. For more information, review the documentation for your specific operating system.

Note
You cannot use the temporary folder function when sending files to an IBM or AS400 host. Any input into this field will be ignored.

Important
The following information is provided as a guide and should not be substituted for information found in AS400 or IBM documentation.


MVS

AS400

There are three methods of naming files and defining their paths when transferring files to and from an AS400 system:

Source: Microsoft

You receive "Failure occurred in parsing the remote folder listing" error when you try to retrieve documents from or send documents to an FTP server

Problem

When you try to retrieve documents from or send documents to an FTP server by using the Microsoft BizTalk Server 2006 FTP adapter, you receive an error message that is similar to the following:

A failure occurred in parsing the remote folder listing.

Cause

You may receive this error message if the following conditions are true:

· The receive handler or the send handler for the BizTalk Server 2006 FTP adapter is configured to use a firewall mode of Passive.

· The target FTP server does not permit passive connections.

Resolution

To resolve this behavior, use one of the following methods:

· Configure the receive handler or the send handler, or both, for the BizTalk Server 2006 FTP adapter to use a firewall mode of Active.

· Configure the target FTP server to permit passive connections.

The FTP adapter can be used to connect to FTP servers on systems ranging from Solaris and Linux to AS/400.

Source: Microsoft

Why does the FTP adapter duplicate or lose data?

If the FTP adapter retrieves a document from the FTP server that is still being written to by the host application, the retrieved document will be incomplete. If the FTP adapter retrieves an incomplete copy of the original document, data duplication or data loss may occur in the following scenarios:

· If the original document is still being written to the FTP server by the host application, the FTP adapter cannot delete the document and will retrieve another copy of the document at the next polling interval that is configured for the receive location. This behavior causes document duplication to occur.

· If the host application has finished writing the document to the FTP server, the document will be deleted. This behavior will cause data loss to occur.

To avoid this behavior, use one of the following methods:

· Configure the host application to write to a temporary folder on the same hard disk as the public FTP folder and to periodically move the contents of the temporary folder to the FTP folder. The temporary folder should be on the same hard disk as the public FTP folder to make sure that the move operation is atomic. An atomic operation is an operation that is functionally indivisible. If you write data to the public FTP folder by using the BizTalk Server 2006 FTP adapter, you can do this by specifying a Temporary Folder property in the FTP Transport Properties dialog box when you configure a send port.

· Configure the FTP receive location to operate within a service window when the host application is not writing data to the FTP server. You can specify the service window when you configure the receive location properties.

Source: Microsoft

How do I enable logging (FTP Adapter) ?

You can enable and configure logging by using the Log folder option in the FTP Transport Properties dialog box when configuring a receive location or send port with the FTP transport type.

Depending upon your FTP server software, you may be able to configure logging for the FTP servers connected to by the BizTalk FTP adapter. Check the documentation for your server to assess your options.

Source: Microsoft

Why isn't my file being picked up by the File receive adapter?

Read-only or system files are not picked up by the File receive adapter and will not be published to the BizTalk Server MessageBox database. Usually the problem is due to a read-only file that was originally copied from a read-only source like a CD-ROM. Files copied from these locations remain read-only until you use Windows Explorer or the MS-DOS attrib.exe utility to turn off the read-only attribute.

If you turn off the read-only flag on a file in a receive location, the File receive adapter will process it.

The File receive adapter also does not pick up files for which it has insufficient rights and files that cannot be exclusively locked. To correct these problems, verify the security settings of the receive location and ensure that any files present are not being held by other processes.

One way to peek at what is happening with your files is to use the Microsoft FileMon utility.

To use FileMon to diagnose File adapter errors

1. Download the FileMon utility from www.sysinternals.com.

2. Install the FileMon utility on the server that hosts the folder or share that is being read from or written to by the File adapter.

3. Launch the FileMon utility and apply a filter to monitor only file accesses that are being made by the BizTalk Host instance that the file adapter handlers are running in (for example, BTSNTSvc.exe).

4. Run the scenario that caused the problem.

5. Disable file monitoring in the FileMon utility, and then save the generated log file to disk.

6. Review the generated log file for any errors.

Source: Microsoft

How to Configure a Base EDI Receive Location

You can set Base EDI receive location adapter variables in the BizTalk Server Administration console.

Note
Before completing the following procedure you must have already added a receive port.

To configure variables for a Base EDI receive location

  1. In the BizTalk Server Administration console, expand BizTalk Server 2006 Administration, expand BizTalk Group, expand Applications, and then expand the application you want to create a receive location in.

  2. In the BizTalk Server Administration console, click the Receive Port node in the left pane. Then in the right pane, right-click the receive port that is associated with an existing receive location or that you want to associate with a new receive location, and then click Properties.

  3. In the Receive Port Properties dialog box, in the left pane, select the Receive Locations option, and in the right pane, double-click an existing receive location or click New to create a new receive location.

  4. In the Receive Location Properties dialog box, in the Transport section next to Type, select EDI from the drop-down list, and then click Configure to configure the transport properties for the receive location.

  5. In the EDI Transport Properties dialog box, do the following:

    Adapter properties

    In this section, select the logical address that you associate with the receive location.

    Use this To do this

    EDI Address (URI)

    The EDI subsystem uses logical addresses to determine the sender and recipient of a document. To determine the sender of a document, the EDI subsystem takes the logical address of the sender of a message, creates a URI from that, and tries to find the receive location record with that URI.

    Note
    The URI for a send port or receive location cannot exceed 256 characters..

    Flags

    In this section, specify the functional acknowledgment settings for this trading partner.

    Use this To do this

    Functional Acknowledgements

    The EDI transport requires you to select a setting for functional acknowledgements.

    Options:

    • Always
    • Never

    Supported document types

    In this section, specify whether to accept unlisted document types.

    Use this To do this

    Accept all unlisted documents

    Specify whether the EDI adapter will accept documents that are not in the list of supported document types at this receive location.

    Options:

    • No
    • Yes

    Supported document types for EDIFACT

    Select from the list of supported EDIFACT document types.

    Supported document types for X-12

    Select from the list of supported X-12 document types.

  6. Click OK.

  7. In the Receive Location Properties dialog box, enter the appropriate values to complete the configuration of the receive location, and then click OK to save the settings.

You receive a "Cant make a connection to BizTalk for document" error message when you send a message through a send port that uses the EDI transport

Problem

When you send a message through a send port that uses the EDI transport, you receive an error similar to the following in the event log:

Event Type: Error Event Source: EDI Subsystem Event Category: BizTalk Server 2006 Event ID: 23 Date: MM/DD/YYYY Time: 4:55:34 PM User: N/A Computer: BIZTALKSERVERDescription: Error encountered: ERROR (1030), interchangenr 10037 : COM_SendMessage(): Cant make a connection to BizTalk for document #10037. Errorcode is -1030 ()

Cause

This can occur when you have not configured and enabled a receive location that uses the EDI transport.

Resolution
To resolve this issue, configure and then enable a receive location that is bound to the EDI transport.

Source: Microsoft

Thursday, August 2, 2007

Indexes in the BAM star-schema database

Some customers have asked why some tables in the star-schema database have indexes and others don't.

As suggested by SQL Books Online “Creating and Using Data Warehouses”, all the dimension tables have the index automatically created on the primary key column “_ID”. (BTW these implicit indexes don’t show up in the Query Analyzer, but you can see them in the Enterprise Manager.) In addition, for the hierarchical dimensions (data dimensions and time dimensions) composite indexes are created on all dimension levels.

One place that doesn’t have index is the fact table. Theoretically, the fact table should be indexed on the composite primary key made up of the foreign keys of the dimension tables. A few special considerations were taken into account before it was decided not to create index on the BAM fact table:

1. Index occurs overhead on data insert (as well as delete & update)

2. Unlike the BAM primary import table which needs to support heavy instance query and real-time aggregation queries, fact table is only queried during the execution of the star-schema transformation stored proc.

3. Unlike dimension table, the entire fact table is truncated before next DTS run. The table is expected to stay relatively small (of course, table size depends on the incoming data volume, the scheduling of cubing DTS and archiving DTS). Indexing small tables may not be optimal because it can take SQL Server longer to traverse the index searching for data than to perform a simple table scan.

4. The star-schema is dynamically created. Customers may know best what additional index they may need on the fact table based on their business data characteristics and DTS scheduling etc.

Source: BAM, BizTalk and Beyond

BAM real-time aggregation vs. scheduled aggregation

BAM supports two types of aggregations: real-time aggregation (RTA) and scheduled aggregation. Some customers have asked what are the differences between them and when to choose what.

The biggest difference is the underling storage. The storage of RTA is a SQL table and the aggregation is updated and maintained by SQL trigger. The event importing and aggregation update are completed in the same transaction, therefore its data latency is negligible and almost real-time. The scheduled aggregation is saved in Olap cubes which need to be processed periodically by the cubing DTS package (btw the Olap cubes and the cubing DTS package are all dynamically created by the BAM Manager command line utility bm.exe).

The biggest advantages of RTA are its zero-latency and low maintenance. Once your new instance data is in the BAM database, the aggregation instantly reflects the new data. And there's no need to run any DTS package, the sql trigger takes care of everything for you automatically. Compared to scheduled aggregation though, RTA doesn't support as many dimensions and measures, and it can only keep relatively short period of data to keep satisfactory query performance. Scheduled aggregation, on the other hand, have little problem handling years of years of enterprise data, and can support much more dimensions and measures. Scheduled aggregation's advantanges don't come without a price -- it needs Analysis Service licence, a star-schema database and scheduling of the cubing DTS. And the new instance data won't make into the cubes until the next cubing DTS run.

After you understand the pros and cons of both aggregations, it's easy to understand their usage scenarios: use RTA for time-critical, small set of Key Performance Indicators (KPI) type of tracking and use scheduled aggregation for historcail/trend analysis which normally involves large volume of data and requires complex cubes.

Source: BAM, BizTalk and Beyond

Installation - Additional Software

The following are some of the additional software components that may or may not be required depending on which components of BizTalk Server you are going to use.

SQL Server 2000 Analysis Services

SQL Server 2000 Analysis Services is the next generation of the OLAP Services component that shipped in SQL Server 7.0. Analysis Services is an easy-to-use, integrated, and scalable set of components that enables you to build multidimensional cubes and provide the application programs with access to the cubes. SQL Server Analysis Services is optional for a BizTalk Server 2004 installation. It is required only if you want to use Health and Activity Tracking (HAT) or Business Activity Monitoring (BAM).

SQLXML 3.0

SQLXML enables XML support for your SQL Server database. It enables developers to bridge the gap between XML and relational data. SQLXML 3.0 is optional for a BizTalk Server 2004 installation. It is required only if you want to use the SQL adapter.

XML Core Services

XML Core Services (formerly known as MSXML, for Microsoft Extensible Markup Language or XML) is an application for processing Extensible Stylesheet Language Transformation (XSLT) in an XML file. XML Core Services is a required piece of software for your BizTalk Server installation.

Microsoft Office XP Tool: Web Components (OWC10)

Microsoft Office Web Components are a collection of Component Object Model (COM) controls for publishing spreadsheets, charts, and databases to the Web, and for viewing the published components on the Web. OWC10 is optional for a BizTalk Server 2004 installation. It is required only if you want to use Health and Activity Tracking (HAT).

Microsoft Office InfoPath 2003

InfoPath 2003 can help teams and organizations efficiently gather the information they need through rich, dynamic forms. The information collected can easily be reused throughout organizations and across business processes because InfoPath 2003 supports industry-standard Extensible Markup Language (XML) using any customer-defined schema. InfoPath is optional for a BizTalk Server 2004 installation. It is required if you plan on configuring Business Activity Services (BAS).

Source: Luke Nyswonger MSDN Blog

Installation - Visual Studio .NET 2003

What is it?

Microsoft Visual Studio .NET 2003 provides a powerful, enterprise team development environment for rapidly building mission-critical applications that target any device and integrate with any platform.

How does it affect my BizTalk Server installation?

Visual Studio .NET is important for the development process in BizTalk Server. To install the BizTalk Server 2004 development tools, you must install Visual Studio .NET 2003 on your computer. The BizTalk Server development tools are based on Visual Studio .NET 2003. Therefore, at a minimum, you must have the Visual C# .NET portion of Visual Studio .NET 2003 installed on your computer before installing BizTalk Server development tools.

You must also install the Visual Studio product documentation for BizTalk Server User Interface (F1) Help to work in the Visual Studio environment.

Source: Luke Nyswonger MSDN Blog

Installation - SQL Server 2000

What is it?

Microsoft SQL Server 2000 is an enterprise-class relational database server capable of efficiently processing high volumes of critical data.

How does it affect my BizTalk Server installation?

The BizTalk Server 2004 engine provides the capability to specify a business process and a mechanism for communicating between applications the business process uses. The BizTalk Server 2004 core engine uses SQL Server 2000 as the main repository for this communication mechanism. SQL Server 2000 is a required piece of the overall architecture.

When you install and configure BizTalk Server 2004, the following SQL Server databases are created:

  • BAM Primary Import database: default database name BAMPrimaryImport
  • Star Schema database: default database name BAMStarSchema
  • HWS Administration database: default database name BizTalkHwsDb
  • BizTalk Tracking database: default database name BizTalkDTADb
  • Configuration database: default database name BizTalkMgmtDb
  • BizTalk MessageBox database: default database name BizTalkMsgBoxDb
  • Rule Engine database: default database name BizTalkRuleEngineDb
  • Trading Partner Management database: default database name TPM
  • Enterprise Single Sign-On database: default database name SSODB
  • BAM Analysis database: default database name BAMAnalysis
  • Tracking Analysis Server database: default database name BizTalkAnalysisDb
  • BizTalk Base EDI database: default database name BizTalkEDIDb

On a single-server installation, the following Windows accounts are created locally when you install and configure BizTalk Server 2004. Items denoted with an asterisk (*) are added as SQL Server accounts:

  • BizTalk Application Users *
  • BizTalk Server Administrators *
  • BizTalk Host Users
  • BizTalk Isolated Host Users *
  • BizTalk Base EDI Users
  • BizTalk BAS Web Services Group *
  • BizTalk BAS Users
  • BizTalk BAS Managers
  • BizTalk BAS Administrators
  • SSO Administrators *
  • SSO Affiliate Administrators
  • EDI Subsystem Users *

The following SQL Server jobs are also created:

  • Backup BizTalk Server
  • CleanupBTFExpiredEntriesJob_BizTalkMgmtDb
  • MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb
  • MessageBox_Message_Cleanup_BizTalkMsgBoxDb
  • MessageBox_Parts_Cleanup_BizTalkMsgBoxDb
  • PurgeSubscriptionsJob_BizTalkMsgBoxDb
  • TrackedMessages_Copy_BizTalkMsgBoxDb
  • TrackingSpool_Cleanup_BizTalkMsgBoxDb
Source: Luke Nyswonger MSDN Blog

Installation - Windows SharePoint Services

What is it?

Before I go any further, let me first state what it is not. Windows SharePoint Services is not SharePoint Portal Server. They are two different yet similar things.

Windows SharePoint Services is a collection of services for Microsoft Windows Server 2003 that you can use to share information, collaborate with other users on documents, and create lists and Web Part pages. SharePoint Portal Server, on the other hand, is a secure, scalable, enterprise portal server built upon Windows SharePoint Services that you can use to aggregate SharePoint sites, information, and applications in your organization into a single, easy-to-use portal. Basically, SharePoint Portal Server is one big cool application built using the Windows SharePoint Services framework.

How does it affect my BizTalk Server installation?

You do not have to install Windows SharePoint Services unless you plan on using Business Activity Services (BAS). BAS is a Web application hosted within Windows SharePoint Services that allows you to interact with trading partners in a collaborative environment. It's also important to note that you could install SharePoint Portal Server 2003 instead of Windows SharePoint Services.

Source: Luke Nyswonger MSDN Blog

Installation - Internet Information Services (IIS)

What is it?

Microsoft Internet Information Services (IIS) is a powerful Web server that provides a highly reliable, manageable, and scalable Web application infrastructure. IIS is optional and must be installed separately after you have installed the base operating system on your computer.

How does it affect my BizTalk Server installation?

Depending on which components of BizTalk Server you use, you will have to configure IIS to work in that environment. Here are the pieces that require IIS:

  • HTTP adapter
  • SOAP adapter
  • Web Services
  • Secure Sockets Layer (SSL) Encryption
  • Windows SharePoint Services
  • Business Activity Services (BAS)
  • Human Workflow Services (HWS)
Source: Luke Nyswonger MSDN Blog