In the last post we looked at key use cases for server-to-server file transfer, where existing solutions tend to fall down, and the evolving requirements that support IT’s need to both maintain a reliable data infrastructure and respond quickly to new business initiatives.

In this post, we’ll look briefly at the modern technology in managed file transfer solutions that drive the ability to meet those evolving requirements.

Underlying the MFT solutions that support reliable and automated point-to-point transfers between servers are a core set of capabilities that are missing in older file transfer technologies.  These are:

• Success Notification: The ability to know definitively that a file has successfully completed a transfer, and when.
• Failure alerts with details: If a problem occurs that interrupts the transfer, when it happens, and what the root cause is likely to be.
• Declarative Automation:  A configuration-driven method (versus complex scripting) for defining and executing file transfers and the associated steps of initiating a transfer, error recovery, and post-processing of the files after they’re transferred.
• Post-processing Actions: The ability to identify commands, scripts, and batches that need to be run from the context of the target system (or the source) following a successful transfer.
• Event-driven initiation: The ability to monitor directories and initiate pre-processing activities and file transfers on the basis of a file showing up in those directories.

Some of these capabilities are often achieved by enhanced (and yes, proprietary) protocols that perform extra steps at the beginning and end of a file transfer.  These allow the transfer endpoints to negotiate the elements of the transfer up front (valid security credentials, target storage definition requirements, post-processing actions that will be run at the end of the transfer, etc.) and perform a handshake at the end of the transfer (Sender, “I sent XXXX bytes.” – Receiver, “Great because I received XXXX bytes.”  or Receiver, “Uh-oh, I only received XXXW bytes, better try again.”).

These capabilities are the building blocks then that are combined to drive real value in to mission-critical server-to-server file transfer scenarios.  Advanced MFT solutions (like Attachmate FileXpress) hit the key areas:

Reliability:

• Native platform support – Windows, UNIX, Linux , IBM i, IBM z/OS
• Error alerting – Send an email or an SNMP trap as soon as a problem occurs
• Error details – Information on the source and location of the error and whether it’s recoverable or not (i.e. insufficient disk space – recoverable, failed user authentication – unrecoverable)
• Automatic retries – Have the MFT software automatically reattempt a recoverable failed transfer after a prescribed waiting period.
• Recoverable work queues – Don’t lose file transfers in a system crash. After system recovery, automatically rebuild the list of file transfers that are queued up to execute.

 Simplified Automation:

• Initiation – The MFT software will start the file transfer process as soon as a user, or script, or application drops a file into a monitored directory
• After the Transfer – The MFT software on the end points will kick off the next step of the process after the transfer is finished.  No scripting or kludgy flag file techniques needed to determine that the transfer is finished.
• Reusability – Configuration-driven file transfer definitions that can be used as templates for future file transfers.

Putting it all together, with MFT solutions server-to-server file transfers become more dependable and automation becomes simpler and quicker to implement.  As such, MFT is helping IT organizations maintain a reliable data infrastructure and respond quickly to new business initiatives.

5 Capabilities Enhancing Server-to-Server File Transfer is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post 5 Capabilities Enhancing Server-to-Server File Transfer appeared first on Data In Motion - a Managed File Transfer blog.

Routine, scheduled and automated file transfers between servers occur in a wide range of use cases in just about every organization. In this post, we will begin a two part series and look at those use cases and the top five factors compelling enhanced server-to-server file transfer.

In the next blog post we will look at the technology requirements to improve server-to server file transfer.

The use cases for automated server-to-server file transfers include: 

• The movement of bulk/batch data from one application to another
• The deployment of web application updates from a staging system to production systems
• The transfer of end-of-day financial and/or inventory data from branch locations into systems hosted in the corporate data center

There are of course many additional uses in any organization.  These are simply some of the more common and generic examples.

5 Factors Impacting Server-to-Server File Transfer

Critical business functions are dependent on these file transfers working consistently, so a reliable and reusable framework for the movement of data between servers within an organization supports the key IT objectives of maintaining a reliable data infrastructure and responding quickly to new business initiatives. 

When server-to-server file transfers fail to support these objectives, it’s typically due to one or more of the 5 following factors: 

• Size/Volume Growth – File sizes and file transfer volumes have grown too a level that’s challenging for existing systems to handle reliably and efficiently.
• Too many solutions – There are a variety of disparate file transfer technologies implemented across the organization’s server-to-server file transfer use cases; there are too many different ways to do the same thing.
• Lack of file transfer governance– Aggregated auditing and reporting needs to be established for all sensitive or regulated data moving within the organization.  See this blog post on file transfer governance.
• High manual effort – Certain file transfers are too critical to leave unmonitored.  Existing technologies can’t provide the reliability and visibility to the critical transfers, requiring IT staff to babysit key transfers.
• Expensive Automation/Integration – The lack of ROI from the effort to automate critical server-to-server file transfers because existing technologies fail to do enough of the work needed for reliability, error recovery and post-transfer activity.  This means that you’re required to code this logic into your scripts and applications, increasing their complexity and making sustainability of those scripts/applications more challenging. 

3 Requirements for Enhanced Server-to-Server File Transfers

Managed File Transfer systems that are optimized for server-to-server file transfer use cases (not all are) support the following requirements that overcome these challenges. 

• Reliably accommodate large files and scale to large volumes.
• Provide a powerful and easy-to-consume interface for file transfer and administrative functions to system administrators and application developers.
• Deliver the visibility necessary to quickly resolve outages, audit the movement of sensitive files and generate reports that document compliance in your handling of regulated data. 

Supporting these requirements is essential in successful and secure server-to-server file transfers that maximize the value of automating routine but vital data transfer.

The next blog post delves into the technical capabilities of advanced MFT solutions that support the above requirements.

5 factors compelling enhanced server-to-server file transfer is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post 5 factors compelling enhanced server-to-server file transfer appeared first on Data In Motion - a Managed File Transfer blog.

“File Transfer Governance” isn’t a term you hear very often.  But it relates to the kinds of rules you deal with in everyday life.

Rules are guiding principles for everything. There are rules to govern the way you conduct yourself at your job, at the gym, as you participate in hobbies and as a citizen. As you go about life, you ask yourself questions to determine if you’re complying with the rules.

The same is true in business. In respect to transferring files, it is critical that organizations are in compliance with the standards – or rules – of their company and any industry requirements. File transfer governance is about monitoring compliance with the policies your organization establishes that enforce those standards. Of course those policies and standards need to be in place first.  Is there a policy establishing the use of a standard for a secure file transfer protocol? Has that secure file transfer protocol standard first been established?

File Transfer Governance Questions to Ask Yourself

We thought it would be helpful to provide you with some quick questions to ask yourself to see where file transfer governance is relevant to your organization and if a managed file transfer (MFT) solution could be an option to help.

1. Is there a corporate policy around encrypting sensitive data during transfer? Do you have procedures in place to ensure compliance with this policy?
• MFT solutions support the secure file transfer protocols that encrypt files in transit. Additionally, they can bridge between insecure and secure protocols making a phased transition possible. MFT solutions often serve as a concentration point for external file transfers, providing file transfer governance by ensuring that no transfer is allowed to occur over an insecure protocol.
2. Is your organization and its employees using multiple solutions for file transfers?
• If yes, then it’s likely each solution has its own approach to configuring security and file transfer governance capabilities, and your team has to learn each one. Consider streamlining your organization’s file transfers with one MFT solution. This can help reduce cost, simplify file transfer governance, and keep the process consistent.
3. Do you leverage “home-grown” applications built on FTP to transfer files?
• This is not uncommon. If yes, look at the potential costs and problems associated with these solutions. Homegrown solutions are typically quite tactical and lack the file transfer governance capabilities built into commercial solutions. Using the insecure FTP protocol, your information could be at risk. This can open the company up to regulatory fines, damage to its image and an increase in business costs when trying to frantically remedy these vulnerabilities as part of a “fire drill” project.
4. What regulatory requirements apply to your business as it relates to protecting data and breach notification? Any international regulations? Does your current solution support compliance with these requirements?
• If your current solution does not support your security policies, then it will be hard-pressed to facilitate compliance with regulatory requirements faced by your organization. Take auditing for instance.  Keeping a record of the secure movement of sensitive files and being able to report in aggregate on file transfer activity over a period of time is a key security control prescribed by file transfer governance and delivered in MFT solutions.

Simplify File Transfer Governance

By leveraging a robust file transfer solution, such as FileXpress, you simplify file transfer governance through such things as consistency, secure protocols and centralized auditing. To help you search for an MFT solution, check out our previous blog post on research tools.

File Transfer Governance? What’s that? is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post File Transfer Governance? What’s that? appeared first on Data In Motion - a Managed File Transfer blog.

Failure is not a term sought after by most. People do not start a project with the intention to fail, but rather to succeed. The same is true for the key file transfers that keep your business moving. Nobody wants to be set up for failure; however, with the wrong tools in place, failure is quite possible.

We’ve come up with the top three classic file transfer “fails” to show what you could be risking.

1. My customer knew the file transfer failed before I did. In interactions and transactions with clients, there is an expectation that promised files will be delivered in a timely manner.  When a customer notices that a file did not make it before the delivering organization does, it can create tension and questions about reliability. Additionally, with FTP (the basis for many home-grown file transfer solutions) there is no alert that a file did not reach its intended destination. So while FTP may be quick and easy to implement, it doesn’t protect you from the potential embarrassment and doesn’t provide a failsafe system that alerts you of an error.
2. My email attachment is too big. The number one file transfer fail that we hear time and time again is an  email attachment that has been blocked from being sent or received because it is too large. This is the way that the majority of people send documents to other people; inside and outside of their organization. Not only is the file transmission not secure, but there is no way to know that the file made it through your email and into your intended target’s inbox. Email attachments also take up inbox space.  “Your inbox is full” messages are so annoying.
3. My IT staff is spending too much time involved manually managing file transfers. When there are multiple file transfer solutions within an organization, it requires a great deal of maintenance. Custom scripts for each solution are hard to maintain and hard to support. This is increasingly difficult as an organization grows in size and the demand increases. Provisioning tasks are often manual, requiring a great deal of time and effort. Wasting resources is a big fail, when resources could be allocated to more pressing tasks.

The good news is these classic fails don’t have to happen to you! Your organization can avoid failure and be prepared for success by implementing a managed file transfer solution, like those offered by Attachmate. Read our previous blog post on tools for evaluating your solution.

Classic File Transfer Fails is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post Classic File Transfer Fails appeared first on Data In Motion - a Managed File Transfer blog.

Everywhere you look people are trying to save a buck or two, but even when you have a goal of reducing operational costs there are some things that aren’t worth shortcuts. In fact, these shortcuts can actually cost you more money in the long run. File transfer is one example of a place you don’t want to cut corners.

We talk a lot about file transfer on the Data in Motion blog, and when it comes to why having a well-designed solution is important, like Deep Throat said in the film “All the President’s Men”, “Follow the money!”

Transferring files seems straightforward; the actual copying process is readily implemented with common networking and server infrastructure. But there are hidden costs that quickly begin to appear without the benefit of a strategic approach to file transfer. Here are three key reasons why failing to make an upfront investment in a managed file transfer solution will result in higher costs down the road:

• Maintenance and support. One of the reasons homegrown file transfer programs can be written so quickly is that they tend to have minimal features. But over time, business requirements change and become more complex. For example, a simple transfer from point A to point B might now need to go to C and D, as well. Expanding a homegrown solution can be costly when you add up the time needed to program and fully test the new solution.
• Opportunity costs. Skilled developers have no shortage of potential projects, which means they don’t have a lot of time to devote to troubleshooting a homegrown file transfer solution that will likely have holes. This means that code will likely fail at inopportune times and developers will be forced to develop, debug and resolve these issues. This isn’t a good use of resources when a developer could otherwise be working on high return-on-investment development initiatives.
• Potential security risks. Data disclosure can have one of the most devastating impacts to a business. Imagine an attacker finding a privileged username and password embedded in a file transfer script. That now-compromised account could be a stepping stone to compromising a server and collecting any data stored on that system. With homegrown solutions, there may be limited or even no protection against unauthorized access which could financially impact an organization in a number of ways.

These hidden costs lurk among homegrown file transfer solutions and controlling these costs is where managed file transfer solutions excel. Imagine being able to easily address these money-draining issues; translating them into long-term savings, and in the long term reducing operational costs.

By providing a more declarative approach to automating file transfers, managed file transfer solutions deliver the flexibility to quickly support new processes. Reusability is key to saving time and money, and this is at the heart of a good managed file transfer offering.

Supportability is another built-in function offered in managed file transfer. What if your file transfers automatically alerted your operations team if there was a problem? Reducing operational costs often means getting more done with fewer IT staff.  What if the file transfer was automatically started again after a failure?  No more lost opportunity cost as your developers tried to figure out where the problem was.

Finally, managed file transfer solutions are built secure. What if your architecture team could quickly set up a secure method of exchanging files over the Internet with business partners and customers?  Imagine a solution that masks the complexity of encryption, protocols, firewalls and DMZs from your developers. 

When it comes to homegrown or even free file transfer solutions, be sure to carefully assess your needs and resources in order to choose a solution that best fits your specific environment. The right fit has the potential to save you more money in the long run.

If you would like more information on how Attachmate’s FileXpress allows your organization to build a strategic and cost-saving file transfer infrastructure, check out our Solution Brief titled “Controlling the Cost of File Transfers“.

Reducing operational costs with managed file transfer is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post Reducing operational costs with managed file transfer appeared first on Data In Motion - a Managed File Transfer blog.

If you’re an IT professional and you’re looking for guidance on how to go about strengthening the security and reliability of file transfer in your organization, then you will want to read The Shortcut Guide to Eliminating Insecure and Unreliable File Transfer Methods written by Dan Sullivan.  This eBook, published by Realtime Publishers and sponsored by Attachmate looks at the challenges experienced when attempting to apply traditional file transfer techniques to today’s larger files and more stringent security standards.

Dan Sullivan is a 20+ year veteran in the IT industry with a history that includes experience in application design, systems architecture, and enterprise security across a broad range of industries, including financial services, manufacturing, government, retail, gas and oil production, power generation, life sciences, and education.  Dan has written and presented extensively about security, systems architecture, infrastructure management, and aligning business and IT strategies. 

So I guess you can say he knows that of which he speaks.

The synopsis of the eBook reads like this:

“Transferring files often appears to be a simple task that is easily scripted. As a result, many organizations are littered with quickly written, insecure and difficult to maintain file transfer scripts. In The Shortcut Guide to Eliminating Insecure and Unreliable File Transfer Methods, author Dan Sullivan outlines the limits and shortcomings of home grown file transfer solutions along with business drivers that are motivating organizations to better manage all file transfers. This book closes with a detailed list of features essential to a dependable file transfer solution and offers guidance on how to deploy such a solution.”

The eBook is written in four chapters, each focused on a particular aspect of file transfer.

• Chapter 1: Understanding the Limits of Traditional File Transfer
• Chapter 2: Analyzing 5 Key Business Drivers for Improving File Transfer
• Chapter 3: Selecting a File Transfer Solution:  7 Essential Requirements
• Chapter 4: Deploying a File Transfer Solution

The eBook can be downloaded from Attachmate here.  Chapter 1 is now available with a new chapter planned each month through July 2010. 

So, check it out, then come back here and tell us what you think.  Comments are welcome.

IT veteran authors new eBook on file transfer security and reliability is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post IT veteran authors new eBook on file transfer security and reliability appeared first on Data In Motion - a Managed File Transfer blog.

My parents have always been real do-it-yourselfers. When I was a kid, every weekend seemed consumed with remodeling some room, or landscaping some portion of our yard. Sometimes they knew what they were doing, sometimes they didn’t. Some of their work has stood the test of time…some of it has had to be replaced.

They were fearless. They were persistent.

I just tried to stay out of the way.

DIY and File Transfer

File transfer is one of those things that many IT organizations have fearlessly developed internal solutions for. I’m not talking just simple point A to point B file movement…but full automation, centralized auditing, and policy enforcement. Real cool stuff.

Frequently the return on these solutions were realized in reduced time to add new file transfers, and reduced time to diagnose when problems occur.

Problems have started to occur with many of those in-house solutions as they have run head-first into challenges imposed by large files, the need for improved security (compliance), and the reality of a reduced ability to maintain or enhance what they have built.

One example…a crumbling foundation

I had a conversation with an IT director of application development for a retail sales company. He had approached us about a problem that he was having with an in-house file transfer solution that his team had built as a departmental project a few years ago. The solution was architected with FTP as a foundation, and it was beginning to show some cracks as some of the transfers of larger files began failing.

Pride of ownership meets reduced staff

I listened to him describe the in-house solution developed by his team, and was impressed at all that it could do and the effort that must have gone into building it. As I described the advantages and capabilities of a managed file transfer solution, I was expecting a certain amount of guardedness driven by a real legitimate pride in his solution. But instead of a “yeah, mine does that too” sort of defensiveness, I encountered only a sense of weariness.

It seems that two things had conspired to generate this attitude.

• The departmental solution had caught on with other departments and as more groups started using it, scalability issues developed, as did new functionality requirements
• His development staff had been cut by more than half

More people were using what his team had built, but there were fewer people on his team to support and enhance it.

“It’s what you do”

The best thing that I heard on that call was something to the effect of, “I like the capabilities you are describing, but what I like the most is that file transfer is what you guys do.” MFT solutions provide an infrastructure with built-in reliability, security, automation, and governance, so that IT departments are relieved from the burden and cost of developing those capabilities themselves.

They’re still do-it-yourselfers, my parents. I love to go over and see their latest landscaping project. I just wouldn’t expect it to transfer 5Gb of private patient health information or batched financial transactions.

The perils of Do-it-Yourself (DIY) file transfer is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post The perils of Do-it-Yourself (DIY) file transfer appeared first on Data In Motion - a Managed File Transfer blog.