Picture this scenario: Your organization has a signed contractual obligation to provide a set of services  to a business partner.  As a part of those services, you must deliver a file to your business partner no later than  4 a.m. on Monday morning. The file is produced by a batch process that is generated over the weekend and then copied to your partner using a file transfer script. Monday morning arrives and you receive an angry phone call that the file never made it to your business partner, and now they are threatening to apply the penalties section of your service level agreement.

Nobody wants that phone call. Unfortunately, that situation is possible when using file transfer protocol (FTP).

Organizations using FTP are often drawn in by the allure of an easy, familiar and inexpensive solution. But there are FTP risks.  As we prepare to close down another year and ring in 2013, it is a good time to evaluate the past year and look for organizational improvements.

We’ve identified the top four FTP risks of 2012 below:

• Reliability. FTP has been seen to have challenges with large files.  Additionally, there is no provision in FTP for determining that an error has occurred in transmission, and alerting an operator that something needs to be done.   A more risk-averse approach to mission-critical file transfers is to use a Managed File Transfer (MFT) solution, which can trigger an alert if a problem arises and even retry the file transfer automatically.
• Security. FTP has no encryption. This means that the data in the transferred files is put at risk and open to network sniffing.  Beyond that, your login credentials (username and password) are also visible to a network sniffer. Imagine the risk you incur if administrative level access to your FTP server becomes available to malicious operator. By contrast, MFT solutions encrypt files as they are sent and keep login information protected.
• Automation. FTP users run the risk of compromising login information when trying to automate the file transfer process. Automating file transfers with FTP is script-based. The system administrator has to write a script that talks to the FTP server, specifies what files to send, and where they should go on the server; however for simplicity, systems administrators will frequently write an FTP username and password into a script. This creates a notable FTP risk. People sharing scripts are also sharing user credentials.  This leaves an organization vulnerable. MFT solutions significantly reduce the need for scripts and offer built-in capabilities for automation. For example, the initiation of the file transfer, the confirmation of the file transfer, the alerting if there are problems with the file transfer, are all done without scripting.
• Auditing and Logging. Another FTP risk comes through in auditing and logging. The challenge with FTP is that it does not provide the level of auditing that organizations need to document compliance with data security regulations. It also does not allow administrators to easily identify when a problem has occurred and help resolve that problem. MFT will help you confirm that a transfer went through and log the details necessary to identify the problem when it doesn’t. This will help you avoid situations like the one described above.

Investing in a more robust solution in the new year could save you and your organization time, money and headaches by avoiding these FTP risks.

What other FTP risks have you identified?

Top FTP Risks of 2012 is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post Top FTP Risks of 2012 appeared first on Data In Motion - a Managed File Transfer blog.

We’ve previously written about the shortcomings of file transfer protocol (FTP) and how organizations using FTP are putting themselves at risk for security and compliance issues. Though FTP is convenient and inexpensive, it can carry devastating consequences.

Despite the warnings, 51 percent of organizations use FTP sites to send and exchange large files.

I recently spoke with Ericka Chickowski for a Dark Reading article, “FTP Ubiquitous and Dangerously Noncompliant,” to comment on security and audit issues with FTP, and shared it is not uncommon for our team to receive requests  from IT managers for a solution to lock down an FTP environment very quickly in response to failed audits. Chickowski writes about how organizations are now subject to audits when using FTP because of some high profile examples where companies have exposed sensitive data.

Chickowski used an example from Yale University where by failing to lock down a database server stored on an FTP server that was found by Google’s search spiders, 43,000 people’s data was left exposed. Acer, a hardware and electronics company, had a similar situation where a hacker accessed information stored on a company FTP, exposing 40,000 customer’s information.

Failed audits, security breaches and unreliability are just a few of the reasons why organizations are realizing the need for a managed file transfer solution (MFT). In the article, Chickowski points to a Forrester report written by Ken Vollmer that suggests the managed file transfer industry is reaping the benefits, “the managed file transfer (MFT) industry measured up to $1.4 billion. As more organizations face the compliance realities of sticking with the old FTP model, that number looks to grow in the coming years.”

For the complete article, read, “FTP Ubiquitous And Dangerously Noncompliant.”

Security and Compliance Risks with FTP is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post Security and Compliance Risks with FTP appeared first on Data In Motion - a Managed File Transfer blog.

“File Transfer Governance” isn’t a term you hear very often.  But it relates to the kinds of rules you deal with in everyday life.

Rules are guiding principles for everything. There are rules to govern the way you conduct yourself at your job, at the gym, as you participate in hobbies and as a citizen. As you go about life, you ask yourself questions to determine if you’re complying with the rules.

The same is true in business. In respect to transferring files, it is critical that organizations are in compliance with the standards – or rules – of their company and any industry requirements. File transfer governance is about monitoring compliance with the policies your organization establishes that enforce those standards. Of course those policies and standards need to be in place first.  Is there a policy establishing the use of a standard for a secure file transfer protocol? Has that secure file transfer protocol standard first been established?

File Transfer Governance Questions to Ask Yourself

We thought it would be helpful to provide you with some quick questions to ask yourself to see where file transfer governance is relevant to your organization and if a managed file transfer (MFT) solution could be an option to help.

1. Is there a corporate policy around encrypting sensitive data during transfer? Do you have procedures in place to ensure compliance with this policy?
• MFT solutions support the secure file transfer protocols that encrypt files in transit. Additionally, they can bridge between insecure and secure protocols making a phased transition possible. MFT solutions often serve as a concentration point for external file transfers, providing file transfer governance by ensuring that no transfer is allowed to occur over an insecure protocol.
2. Is your organization and its employees using multiple solutions for file transfers?
• If yes, then it’s likely each solution has its own approach to configuring security and file transfer governance capabilities, and your team has to learn each one. Consider streamlining your organization’s file transfers with one MFT solution. This can help reduce cost, simplify file transfer governance, and keep the process consistent.
3. Do you leverage “home-grown” applications built on FTP to transfer files?
• This is not uncommon. If yes, look at the potential costs and problems associated with these solutions. Homegrown solutions are typically quite tactical and lack the file transfer governance capabilities built into commercial solutions. Using the insecure FTP protocol, your information could be at risk. This can open the company up to regulatory fines, damage to its image and an increase in business costs when trying to frantically remedy these vulnerabilities as part of a “fire drill” project.
4. What regulatory requirements apply to your business as it relates to protecting data and breach notification? Any international regulations? Does your current solution support compliance with these requirements?
• If your current solution does not support your security policies, then it will be hard-pressed to facilitate compliance with regulatory requirements faced by your organization. Take auditing for instance.  Keeping a record of the secure movement of sensitive files and being able to report in aggregate on file transfer activity over a period of time is a key security control prescribed by file transfer governance and delivered in MFT solutions.

Simplify File Transfer Governance

By leveraging a robust file transfer solution, such as FileXpress, you simplify file transfer governance through such things as consistency, secure protocols and centralized auditing. To help you search for an MFT solution, check out our previous blog post on research tools.

File Transfer Governance? What’s that? is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post File Transfer Governance? What’s that? appeared first on Data In Motion - a Managed File Transfer blog.

Failure is not a term sought after by most. People do not start a project with the intention to fail, but rather to succeed. The same is true for the key file transfers that keep your business moving. Nobody wants to be set up for failure; however, with the wrong tools in place, failure is quite possible.

We’ve come up with the top three classic file transfer “fails” to show what you could be risking.

1. My customer knew the file transfer failed before I did. In interactions and transactions with clients, there is an expectation that promised files will be delivered in a timely manner.  When a customer notices that a file did not make it before the delivering organization does, it can create tension and questions about reliability. Additionally, with FTP (the basis for many home-grown file transfer solutions) there is no alert that a file did not reach its intended destination. So while FTP may be quick and easy to implement, it doesn’t protect you from the potential embarrassment and doesn’t provide a failsafe system that alerts you of an error.
2. My email attachment is too big. The number one file transfer fail that we hear time and time again is an  email attachment that has been blocked from being sent or received because it is too large. This is the way that the majority of people send documents to other people; inside and outside of their organization. Not only is the file transmission not secure, but there is no way to know that the file made it through your email and into your intended target’s inbox. Email attachments also take up inbox space.  “Your inbox is full” messages are so annoying.
3. My IT staff is spending too much time involved manually managing file transfers. When there are multiple file transfer solutions within an organization, it requires a great deal of maintenance. Custom scripts for each solution are hard to maintain and hard to support. This is increasingly difficult as an organization grows in size and the demand increases. Provisioning tasks are often manual, requiring a great deal of time and effort. Wasting resources is a big fail, when resources could be allocated to more pressing tasks.

The good news is these classic fails don’t have to happen to you! Your organization can avoid failure and be prepared for success by implementing a managed file transfer solution, like those offered by Attachmate. Read our previous blog post on tools for evaluating your solution.

Classic File Transfer Fails is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post Classic File Transfer Fails appeared first on Data In Motion - a Managed File Transfer blog.

Despite stricter compliance guidelines and an increased awareness of security breaches, a Formtek blog post showcases the prevalence of FTP and the need for a more secure solution.

In its post called, “Document Security: Managed File Transfers Enhance Standard FTP,” FormTek highlights survey data showcasing  that  more than 77 percent of companies still use FTP to transfer files.

The numbers reveal that a great deal of data in the enterprise is flowing in large batch files with no encryption. Formtek says that of those companies that don’t currently use FTP, 40 percent reveal they’d like to set up an FTP site to allow data exchange. The problem is, FTP has been around for 40 years and offers no reliable way to securely transfer files.

FTP does not include any type of encryption for transmitting data. The survey reveals that when given a choice between standard FTP versus secure techniques like SFTP, FTPS and HTTPS, only 20 percent said they would choose FTP.  The Formtek post highlights Managed File Transfer (MFT) as technology that addresses FTP security concerns, noting, “MFT solutions include security components to the process of the data transfer, making it more secure.” In addition, MFT offers increased reliability.

Formtek also notes that the survey found the top three challenges to companies that are using file exchange technologies are:

• 49 percent – security
• 30 percent – file size
• 24 percent – file management

These findings are similar to an Attachmate survey that found 53 percent of respondents consider security their greatest challenge when working with file transfers.  This again points to the need for a cost-effective alternative to FTP to securely manage file transfers that can address these top concerns to effectively transfer files of any size, securely and reliably.

To read this full article, visit the FormTek blog.

MFT Enhances FTP is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post MFT Enhances FTP appeared first on Data In Motion - a Managed File Transfer blog.

The market for managed file transfer solutions is constantly changing. It is important for vendors and consumers to be savvy about these driving forces to anticipate their organization’s needs. Forrester recently came out with a report, “Market Overview: Managed File Transfer Solutions,” in which Analyst Ken Vollmer interviewed 20 MFT vendors, including Attachmate, to gather industry information and perspective.

Vollmer found the market is growing rapidly. One of the largest drivers of the growth is the increased awareness about the limitations of file transfer protocol – or FTP. Both internal and external auditors are demanding “better security and improved governance over information that leaves the enterprise.” Vollmer says that MFT solutions provide an effective option for addressing this need. Vollmer’s report suggests most MFT vendors are targeting three specific sectors, which have seen an increased need for MFT solutions. The financial, healthcare and government sectors are all seeing an increased need for a solution that addresses increased security to maintain privacy and critical information.

Forrester believes MFT revenue grew about 15 percent year-over-year in 2010 and expects this growth rate will continue for the next three years as more and more organizations realize the value of MFT solutions and update their infrastructures.

The report concludes that the trend of traditional FTP being replaced with more robust MFT solutions will continue and that those FTP solutions will “no longer be adequate for meeting file transfer needs.” I’ve written about some of the limitations with FTP in previous posts.

For more information, including interesting survey data, you can check out the full Forrester report, “Market Overview: Managed File Transfer Solutions.”

A Look at the Managed File Transfer Market is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post A Look at the Managed File Transfer Market appeared first on Data In Motion - a Managed File Transfer blog.

“Please get that file to me.” I said. “Sure,” he said, “I’ll put it up on my ftp server. You get a client that does ftp to a server, and you’ll be able to pull it down.” Simple and quick? Sure. Secure and scalable?  Definitely not.

It’s kind of like shopping for something big, like a car. As a consumer, when you’re looking to make a large purchase, you make decisions by evaluating your need, ability and objectives. Shopping around is a critical aspect of the process. You wouldn’t buy the first sports car you saw on the lot when you have five children and two dogs, right?

The same is true for selecting the right solution for transferring files within your organization. Many companies recognize the importance file transfer technology plays in their enterprise, and have adopted strategic solutions that meet a broad set of requirements around reliability, scalability and security. In other companies, file transfer is synonymous with “ftp to a server.” The latter may work well for ad hoc transfers within the corporate network between somewhat technical users, but it doesn’t take into account the broader scope of use cases and requirements.

Where does “ftp to a server” play well?  How does that contrast with the capabilities delivered in a more comprehensive managed file transfer (MFT) solution?

Consider the following contrast between FTP and MFT:

Ultimately each organization must prioritize their needs. But I caution each business to carefully evaluate the type of information they are transferring and what makes the most sense. Nobody wants to be left hauling a family of five in a BMW Z3.

Is FTP the Right Solution for Your Organization? is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post Is FTP the Right Solution for Your Organization? appeared first on Data In Motion - a Managed File Transfer blog.

They say a picture is worth a thousand words.  Probably.  Back in 1987, the youth of America had been hearing the slogan “just say ‘no’ to drugs” for five years.  It had lost most of its value and according to the Partnership for a Drug-Free America™, “attitudes regarding drug use evolved from ‘acceptable and harmless’ to ‘addictive and dangerous.’” (The Partnership’s “Fried Egg TV Message”)

It must be human nature or something.  We hear statements like “wearing a seatbelt can save your life”, or “flossing your teeth can prevent gum disease”.  And yet we often let convenience trump wise precaution. 

Here’s another statement you hear a lot from secure file transfer vendors like Attachmate.

“FTP is not a secure file transfer protocol”

What does that mean exactly?

The FTP protocol specification makes no provision for the encryption of data in motion.  This means that while reliable and pervasive (convenient), FTP communications have some nonsecure behaviors including:

• Usernames and Passwords are not encrypted as they are communicated over the network.
• The contents of the file transferred (the data payload) are transmitted without encryption from FTP over the network.

If you are interested in seeing how simple it is to leverage a readily-accessible packet sniffing tool (like Wireshark) to capture a username and password from an FTP connection, check out the video “Password Sniffing with Wireshark” by Laura Chappell, founder of Wireshark University.

In addition, FTP has a few other shortcomings that more modern and secure file transfer approaches offer, including:

• In FTP, it’s very difficult to know that a file transfer has been successfully and accurately completed.  This makes it difficult when automating file transfers, say via scripting, because you have to go outside of the file transfer command to ensure that the file transfer is finished and successful.
• FTP logging is limited and makes it difficult for IT operations folks to know when a problem has occurred and what the source of the problem might be.
• FTP doesn’t care what happens to the file after it arrives at its target location.  No one transfers a file because it is fun to do.  They need something to happen to the file after it gets where it has been transferred to. (In the managed file transfer world, we call this “post-processing”.)

So you could characterize FTP’s challenges as being in the areas of security, reliability, operational support, and automation.

Alright, rather than repeating the slogan “just say ‘no’ to FTP” in future blog posts, I’ll simply leave you with this little video.

This is your brain on FTP is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post This is your brain on FTP appeared first on Data In Motion - a Managed File Transfer blog.

I’ve always heard stories about restaurants that throw challenges at their patrons claiming, “if you eat the whole thing, you don’t have to pay for it.” 

I’ve most recently heard about the “World Famous” 72-ounce (about 2 kilograms) steak dinner that one can get for free (if eaten in 1 hour).  This feast is offered by the Big Texan Steak Ranch in Amarillo, Texas.

To quote their web site, “Many have tried.  Many have failed.”

Just as 72-ounce steaks will provide a significant challenge to the gastro-intestinal systems of the average human being, so too do large files when one attempts to transfer them via legacy file transfer systems.

One of the most pervasive file transfer methods in use is good old FTP.  FTP, which stands for “file transfer protocol” has its roots in specifications that go back as far as early 1971.  While some may argue that the FTP protocol itself has nothing inherently in it that prohibits the transfer of very large files, these large files often present a real challenge when transferred between FTP clients and servers.  Just search on the Internet for “FTP 4Gb”.  The problems frequently lay instead with design limitations of the software implementing the FTP protocol (often old clients or servers, written long ago), or the platforms on which the software is running.

The SFTP transfer protocol, an extension to the secure shell (SSH) protocol provides for more reliability while also adding security.  It does this by means of a message authentication code (MAC) that is computed for each TCP/IP packet.  This provides for data integrity which helps ensure that the file is transferred with complete accuracy and isn’t corrupted (or intentionally altered) in transit.

As managed file transfer solutions have evolved, more proprietary protocols have been developed in the interest of driving additional reliability into large file transfers.  Some of these protocols implement key reliability capabilities:

• Notification that a file transfer has been accurately completed
• Detailed feedback on errors and their causes
• Checkpoint control packets

Out of these capabilities, implemented in the proprietary protocol, file transfer recovery features have been designed that really drive reliability into the file transfer process.  These include:

• Automatic Retries – if the file transfer client knows that an error is one that can be recovered from, it can reattempt a file transfer after a pre-defined waiting period.
• Checkpoint/Restart – Checkpoint control packets tell the sender and receiver how much of the file has been transferred so far.  In the event of a failed transfer, the file transfer can resume at the last checkpoint, rather than restart at the beginning of the file.

In addition to the reliability benefits gained through proprietary protocols, some managed file transfer solutions implement the concept of a work queue in which file transfers are lined up, waiting for their chance to execute.  Work queue recovery is the process by which those jobs are retained in the event of a system crash and loaded back into the queue upon system restart.

Maybe you only push out a 4Gb file with the same frequency that you would eat a 72-ounce steak.  In other cases, you may be experiencing reliability issues on much smaller files.  Regardless, having the added reliability of a managed file transfer solution that supports both open and proprietary protocols will make a big difference when the file has to get there, complete and unaltered.

“I can’t believe I ate the whole thing!” is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post “I can’t believe I ate the whole thing!” appeared first on Data In Motion - a Managed File Transfer blog.

My parents have always been real do-it-yourselfers. When I was a kid, every weekend seemed consumed with remodeling some room, or landscaping some portion of our yard. Sometimes they knew what they were doing, sometimes they didn’t. Some of their work has stood the test of time…some of it has had to be replaced.

They were fearless. They were persistent.

I just tried to stay out of the way.

DIY and File Transfer

File transfer is one of those things that many IT organizations have fearlessly developed internal solutions for. I’m not talking just simple point A to point B file movement…but full automation, centralized auditing, and policy enforcement. Real cool stuff.

Frequently the return on these solutions were realized in reduced time to add new file transfers, and reduced time to diagnose when problems occur.

Problems have started to occur with many of those in-house solutions as they have run head-first into challenges imposed by large files, the need for improved security (compliance), and the reality of a reduced ability to maintain or enhance what they have built.

One example…a crumbling foundation

I had a conversation with an IT director of application development for a retail sales company. He had approached us about a problem that he was having with an in-house file transfer solution that his team had built as a departmental project a few years ago. The solution was architected with FTP as a foundation, and it was beginning to show some cracks as some of the transfers of larger files began failing.

Pride of ownership meets reduced staff

I listened to him describe the in-house solution developed by his team, and was impressed at all that it could do and the effort that must have gone into building it. As I described the advantages and capabilities of a managed file transfer solution, I was expecting a certain amount of guardedness driven by a real legitimate pride in his solution. But instead of a “yeah, mine does that too” sort of defensiveness, I encountered only a sense of weariness.

It seems that two things had conspired to generate this attitude.

• The departmental solution had caught on with other departments and as more groups started using it, scalability issues developed, as did new functionality requirements
• His development staff had been cut by more than half

More people were using what his team had built, but there were fewer people on his team to support and enhance it.

“It’s what you do”

The best thing that I heard on that call was something to the effect of, “I like the capabilities you are describing, but what I like the most is that file transfer is what you guys do.” MFT solutions provide an infrastructure with built-in reliability, security, automation, and governance, so that IT departments are relieved from the burden and cost of developing those capabilities themselves.

They’re still do-it-yourselfers, my parents. I love to go over and see their latest landscaping project. I just wouldn’t expect it to transfer 5Gb of private patient health information or batched financial transactions.

The perils of Do-it-Yourself (DIY) file transfer is a post from: Data In Motion - a Managed File Transfer blog

Share and Enjoy

The post The perils of Do-it-Yourself (DIY) file transfer appeared first on Data In Motion - a Managed File Transfer blog.