学步园

Problem:

Managing files web applications should be quick and easy and most importantly, consistent. The traditional way to store files is on the file system or within a RDBMS (SQL SERVER, Oracle, MySql) or on SharePoint. However what if you were developing several applications that shared a million files and each was 5MB. Which solution would you choose? This is commonly the case for large content driven sites (Music Stores, video sharing sites, content management systems, etc).

Solution:

In providing a solution I would implement a SOA based File Service, and this article will guide you to making your own File Service. The reasons that I believe a File Service works are:

• Single point of storage for data
• Can be used by multiple applications
• Single source for security
• Easy to de-duplicate files
• Clear separation between binary data and metadata
• Efficient file system usage ensuring that director retrieval is fast
• Efficient IO usage using multiple disks (per share)

Data Structure:

Firstly we need to create a table to store our file information. Most columns on the table shown in Figure 1 are self explanatory and for possible enhancements see the "Future Enhancements" section below.

Figure 1

Solution Structure:

1. Create a new Solution in Visual Studio 2008, targeting the .NET Framework 3.5
2. Create a new Class Library called "FileServiceCore"
   1. Add a new class called "FileHandlerFactory.cs"
   2. Add a new class called "FileHandlerLinq.cs"
   3. Add a new class called "FileDistributor.cs"
   4. Add a new LINQ to SQL Classes called "FileService.dbml"
   5. Using the Server Explorer create a new connection to the database that contains the "Files" table described above. You should be able to drag and drop the table onto the designer.
3. Create a new Website called "FileService"
   1. Add a project reference to the FileServiceCore
   2. In the website root add a new generic handler called "ImageHandler.ashx"
   3. In the website root add a new generic handler called "DownloadHandler.ashx"
4. Create a new website called "SampleWebsite"
   1. Add a project reference to the FileServiceCore

Your solution should now look like figure 2 below.

Figure 2

Logic:

Now that you have created your Data Structure and Visual Studio solution we will need to add the application logic to each of the classes (e.g. FileHandlerFactory.cs).

FileDistributor.cs

The FileDistributor class is responsible for distributing files to the file system. It contains a single static method that calculates a network file path for a FileId.

namespace FileServiceCore
{
public static class FileDistributor
{
/**//// <summary>
/// We use a static variable to tell the system which path to use
/// This ensures the file distributor acts as similar to a Round robin DNS
/// </summary>
private static volatile int pathId = 0;
/**//// <summary>
/// Gets the distributed file path.
/// </summary>
/// <param name="fileId">The file id.</param>
/// <returns></returns>
public static string GetDistributedFilePath(int fileId)
{
// normally you would store these in a config file or a database, however in keeping this example
// simple I have hard coded the network share paths
string[] networkSharePaths = new string[] {
@"C:\Temp\FileService\Share1\",
@"C:\Temp\FileService\Share2\",
@"C:\Temp\FileService\Share3\"};
// we use an internal variable to ensure there are no multithreading issues
int innerPathId = pathId;
// check that we have a valid path
pathId += 1;
if (pathId >= networkSharePaths.Length)
{
pathId = 0;
}
// check that we have a valid path for the internal variable, as we don't want
// to have a situation where innerPathId doesn't reference a valid network share paths
// array index
if (innerPathId >= networkSharePaths.Length)
{
innerPathId = 0;
}
// check the root path exists and creates the directory if it does not
if (!System.IO.Directory.Exists(networkSharePaths[innerPathId]))
{
System.IO.Directory.CreateDirectory(networkSharePaths[innerPathId]);
}
// pad the fileId to 9 places which allows for about 999,999,999 files
string paddedFileId = fileId.ToString().PadLeft(9, '0');
// check main folder
string mainFolder = System.IO.Path.Combine(networkSharePaths[innerPathId], paddedFileId.Substring(0, 3));
if (!System.IO.Directory.Exists(mainFolder))
{
System.IO.Directory.CreateDirectory(mainFolder);
}
// check the next folder down
string fileFolder = System.IO.Path.Combine(mainFolder, paddedFileId.Substring(3, 3));
if (!System.IO.Directory.Exists(fileFolder))
{
System.IO.Directory.CreateDirectory(fileFolder);
}
// get the final file name (this will be in the xxx.extension format)
string finalFileName = paddedFileId.Substring(6);
// the result should be similar to the following
//  C:\Temp\FileService\Share1\
//      000
//          000
//              001.txt
//              002.txt
return System.IO.Path.Combine(fileFolder, finalFileName);
}
}
}

FileHandlerLinq.cs

The FileHandlerLinq class is a static class that each calling application can use to Insert/Select files from the Files table. For the purpose of this example update and deletes have been omitted.

Code
using System.Linq;
namespace FileServiceCore
{
public static class FileHandlerLinq
{
/**//// <summary>
/// Selects the specified file by a FileId.
/// </summary>
/// <param name="fileId">The file id.</param>
/// <returns><see cref="File"/></returns>
[System.ComponentModel.DataObjectMethod(System.ComponentModel.DataObjectMethodType.Select)]
public static File Select(int fileId)
{
using (FileServiceDataContext db = new FileServiceDataContext())
{
return (from z in db.Files
where z.FileId.Equals(fileId)
select z).FirstOrDefault();
}
}
/**//// <summary>
/// Inserts a record into the Files table and moves the file to a network share.
/// </summary>
/// <param name="fullFileName">Full name of the file.</param>
/// <param name="contentType">Type of the content.</param>
/// <returns><see cref="File"/></returns>
public static File Insert(string fullFileName, string contentType, System.IO.Stream inputStream)
{
using (FileServiceDataContext db = new FileServiceDataContext())
{
// get the file properties (so we can get the mime type, length, etc)
System.IO.FileInfo fileInfo = new System.IO.FileInfo(fullFileName);
File entity = new File();
entity.ContentType = contentType;
entity.ContentLength = (int)inputStream.Length;
entity.FileName = fileInfo.Name;
entity.CreatedDate = System.DateTime.Now;
// we set the network path to empty as it will be generated from 
// the database identity seed column "FileId"
entity.NetworkPath = string.Empty;
// inserting the record will update the entity object
db.Files.InsertOnSubmit(entity);
db.SubmitChanges();
// update the new path to be the correct location
entity = (from z in db.Files
where z.FileId.Equals(entity.FileId)
select z).FirstOrDefault(); ;
entity.NetworkPath = string.Concat(FileDistributor.GetDistributedFilePath(entity.FileId), fileInfo.Extension);
// convert the stream into a series of bytes
byte[] bytes = new byte[(int)inputStream.Length];
inputStream.Read(bytes, 0, (int)inputStream.Length);
// write the bytes to our network path
System.IO.File.WriteAllBytes(entity.NetworkPath, bytes);
// update the database with the new network path
// this will be in the format of \\FileShare\xxx\xxx\xxx.extension
db.SubmitChanges();
return entity;
}
}
}
}

FileHandlerFactory.cs

The FileHandlerFactory is a simple factory class that determines if the browser should handle the file or if the file should be downloaded by the client as an attachment. In this example the factory is simple and should be customized for your own needs.

using System;
using System.Web;
namespace FileServiceCore
{
public static class FileHandlerFactory
{
/**//// <summary>
/// Transmits the file.
/// </summary>
/// <param name="context"><see cref="HttpContext"/>.</param>
/// <param name="fileId">The file id.</param>
public static void TransmitFile(HttpContext context, int fileId)
{
TransmitFile(context, fileId, false);
}
/**//// <summary>
/// Transmits the file.
/// </summary>
/// <param name="context"><see cref="HttpContext"/>.</param>
/// <param name="fileId">The file id.</param>
/// <param name="forceAsAttachment">if set to <c>true</c> [force as attachment].</param>
public static void TransmitFile(HttpContext context, int fileId, bool forceAsAttachment)
{
File file = FileHandlerLinq.Select(fileId);
if (file != null)
{
// parse the file name and get the extension, we use a ToUpperInvariant 
// as there are some characters that cannot be converted to lower case
string extension = file.NetworkPath.Substring(
file.NetworkPath.LastIndexOf(
".", StringComparison.OrdinalIgnoreCase
)
).ToUpperInvariant();
if (forceAsAttachment)
{
extension = string.Empty;
}
// in keeping this example simple we simply parse the file extension to determine if
// we should set the file as an attachment or send it back to the browser
switch (extension)
{

返回

【上篇】Android如何播放GIF动画
【下篇】怎么侦测USB端口上有设备插拔

作者: 305495017sostar

• 该日志由 305495017sostar 于11年前发表在综合分类下，最后更新于 2013年05月15日.
• 转载请注明: Building a File Service | 学步园 +复制链接

抱歉!评论已关闭.