PowerPivot Download

(1) Copy.asmx file upload.

Byte array can have contiguous memory allocation problems on the server.

(2) WebDav (WebClient) file upload

No metadata can be uploaded.

(3) RPC (FrontPage) file upload

However, it does not support sending any metadata long with the file content. This can be a major problem if the document library has multiple content types, so the new file will be put into the document library with the default content type. Another big issue is if the default content type has required fields. The file will remain checked out until the fields are populated. This prevents other users from seeing the document or from being returned in any searches.  It is a great solution if you are just bulk migrating data from an external data store to SharePoint. You more than likely will have to do extra work afterwards. Adding metadata after uploading will also cause the creation of extra versions of the document being created unnecessarily. The fact that it does not use the soap protocol but straight http makes it more scalable than the copy web service. Unfortunately, it still suffers from the fact that it uses a byte array to upload the file. So sooner or later you will run into “out of memory “ exceptions.  So how can I create a folder before using WebDav?  You can use the lists web service to accomplish this:

Field engineer on Partition Processing dimensions in SSAS.

http://blogs.msdn.com/b/sqlcat/archive/2007/03/05/ssas-partition-slicing.aspx

Troubleshooting partition queries.

In planning the partition scheme, you must decide what filegroup or filegroups that you want to put your partitions on. The primary reason for placing your partitions on separate filegroups is to make sure that you can independently perform backup operations on partitions. This is because you can perform backups on individual filegroups. For more information, see Backing Up and Restoring Databases in SQL Server.

http://blog.sqlauthority.com/2008/01/25/sql-server-2005-database-table-partitioning-tutorial-how-to-horizontal-partition-database-table/

Good overview of steps for a database on two filegroups followed by a partition that uses the two file groups for the two partitions and creation of index.

http://sqlblog.com/blogs/erin_welker/archive/2008/02/10/partitioning-enhancements-in-sql-server-2008.aspx

Partitioning enhancements for SQL Server 2008 around the allocation of threads to active partitions in a query.

Note that this makes it all the more important to allocate partitions to filegroups that are spread across many disk spindles, allowing the query on a given partition to be as efficient as possible.

But wait, there’s another improvement to partition query behavior in SQL Server 2008. SQL Server 2005 only allows lock escalation to the table level. Let’s say your query eliminates 75% of a table’s partitions, but scans all of the remaining partitions (1/4 of the table). If SQL Server decides to escalate the lock, all other queries will be locked out even if they are querying completely different partitions. SQL Server 2008 provides a table option to override this default behavior. Note that the default is still to escalate to table locks (at least for now) so this option will need to be changed to take advantage of partition-level lock escalation. Find out more about this, like I did, on über blogger Paul Randall’s post.

http://msdn.microsoft.com/en-us/library/cc280449.aspx

Partitions have compression settings.

http://www.codeproject.com/Articles/43629/Top-10-steps-to-optimize-data-access-in-SQL-Server

Partitioning best practices

• Consider partitioning big fat tables into different file groups where each file inside the file group is spread into separate physical disks (so that the table spans across different files in different physical disks). This would enable the database engine to read/write data operations faster.
• For history data, consider partitioning based on “Age”. For example, suppose a table has order data. To partition this table, use the Order date column to split the table so that a partition is created to contain each year’s sales data.

• Add user defined file groups to the database

Use the following SQL command to create a file group:

Collapse | Copy Code

ALTER DATABASE OrderDB ADD FILEGROUP [1999]

ALTER DATABASE OrderDB ADD FILE (NAME = N'1999', FILENAME
= N'C:\OrderDB\1999.ndf', SIZE = 5MB, MAXSIZE = 100MB, FILEGROWTH = 5MB) TO
FILEGROUP [1999]
http://www.simple-talk.com/sql/database-administration/partitioned-tables-in-sql-server-2005/
Adding and removing partitions.

Partitioned tables and Indexes

If the index contains the partitioning column then the index is referred to as being ‘aligned’ with the table.

If the index uses the same partitioning scheme as the table and is in the same filegroup then the index must be aligned with the table.

For a non-clustered non-unique index the partitioning column can be included to align the index rather than being indexed.

I think it is best to always explicitly include the partitioning column in your indexes.

Partitioning on multiple columns

Although the partitioning column must be a single column, it does not need to be numeric and it can be calculated so that the range can include multiple columns. For instance it is common to partition on datetime data by month. This will work well, because that data is usually in a single column, but what do you do if you have data for multiple companies and you also want to partition by company? For this you could use a computed column for the partitioning column. This will create a computed column using the ‘company id’ and ‘order month’ which is then used for the partitions. It will partition three companies for the first three months of 2007.

The computed column must be ‘persisted’ to form the partitioning column.

Monthly Data – the sliding range

A common requirement is to partition by month. This means that new month partitions need to be added and possibly old data partitions removed. I will describe the process for the addition of a new partition for later data, to remove an old partition the process is the same except that you swap out two partitions, merge the range and swap in a single table.

We create a partitioned table for data by OrderDate month

This will give four partitions…

Therefore the data will be split intopartitions by month, and we insert some test data:

The primary reason is performance. When you run out of IOPS capacity on your primary filegroup disk drive, you’ll need to expand onto a second filegroup to split IOPS over multiple disks/LUNs depending on storage config.

EDIT: Brad Wilson made a good comment regarding SSDs. If you’re using a composite SSD/SATA/FC storage system, you may want to have different filegroups on different types of storage. You can then put your extreme IOPS requirement tables onto SSD filegropus, while history/stats tables may be stored on cheap SATA filegroups.

The only thing I would add is that you can also frequently off-load indexes from heavily accessed tables to their own spindles/filegroups to boost async/read-ahead performance.

I would also point out there is a recoverabiltiy / data availability aspect to this question as well. By using multiple file groups, and not placing any user defined objects on the primary file group, you have more flexibility in enabling online restores. this allows piecemeal restore at the file group level.

http://www.codeproject.com/Articles/43629/Top-10-steps-to-optimize-data-access-in-SQL-Server

File group

Database files are logically grouped for better performance and improved administration on large databases. When a new SQL Server database is created, the primary file group is created and the primary data file is included in the primary file group. Also, the primary group is marked as the default group. As a result, every newly created user object is automatically placed inside the primary file group (more specifically, inside the files in the primary file group).

If you want your user objects (Tables/Views/Stored Procedures/Functions, and others) to be created in a secondary data file, then:

• Create a new file group and mark that file group as Default.
• Create a new data file (.ndf file) and set the file group of this data file to the new file group that you just created.

After doing this, all subsequent objects you create in the database are going to be created inside the file(s) in the secondary file group.

Please note that, Transaction log files are not included in any file group.

File/ File group organization best practices

When you have a small or moderate sized database, then the default file/ file group organization that gets created while creating the database may be enough for you. But, when your database has a tendency to grow larger (say, over 1000 MB) in size, you can (and should) do a little tweaking in the file/file group organizations in the database to enhance database performance. Here are some of the best practices you can follow:

• The primary file group must be totally separate, and should be left to have only system objects, and no user defined object should be created on the primary file group. Also, the primary file group should not be set as the default file group. Separating the system objects from other user objects will increase performance and enhance the ability to access tables in cases of serious data failures.
• If there are N physical disk drives available in the system, then try to create N files per file group and put each one in a separate disk. This will allow distributing disk I/O load over multiple disks, and will increase performance.
• For frequently accessed tables containing indexes, put the tables and the indexes in separate file groups. This would enable reading the index and table data faster.
• For frequently accessed table containing Text or Image columns, create a separate file group and put the text, next, and image columns in that file group on different physical disks, and put the tables in a different file group. This would enable faster data retrieval from the table with queries that don’t contain text or image columns.
• Put the transaction log file on a different physical disk that is not used by the data files. The logging operation (Transaction log writing operation) is more write-intensive, and hence it is important to have the log on the disk that has good I/O performance.
• Consider assigning “Read only” tables into a file group that is marked as “Read only”. This would enable faster data retrieval from these read only tables. Similarly, assign “Write only” tables in a different file group to allow for faster updates.
• Do not let SQL Server fire the “Auto grow” feature too often because it is a costly operation. Set an “Auto grow” increment value so that the database size is increased less frequently (say, once per week). Similarly, do not use the “Auto shrink” feature for the same reason. Disable it, and either shrink the database size manually, or use a scheduled operation that runs in a timed interval (say, once a month).

At microsoft’s recomendations we changed the code to update BOTH the fields for an MMS field as shown below :-

batchElement.InnerXml = “<Method ID=’1′ Cmd=’Update’>”

+ “<Field Name=’ID’>3</Field>”

+ “<Field Name=’Title’>Modified through lists.asmx web service 1</Field>”

+ “<Field Name=’Country’>3;#Germany</Field>”

+ “<Field Name=’Continent’>2</Field>”

+ “<Field Name=’ContinentTaxHTField0′>0;#Northamerica|d511f3c7-377f-480f-aff6-beebecd3c675</Field>”

+ “</Method>”;

I also found that the pointer “<Field Name=’Continent’>2</Field>” could be set to 0 if the term was not already in the hidden list. ie “<Field Name=’Continent’>0</Field>”

SQL Server Agent and proxy accounts.

Conspicuous by its absence is the Transact-SQL script (T-SQL) subsystem. T-SQL jobsteps always run in the security context of the job owner, and there’s no override in SSMS to change that. You can manually configure the job step, using the sp_add_jobstep system stored procedure, passing the database_user_name parameter to impersonate a database user for a jobstep, but of course you’ll need security rights to impersonate a database user to do so.

In order for a proxy account to work correctly, the account must have the “Log on as a batch job” (seBatchLogonRight) assigned to it by a Windows administrator (for example, in the Local Security Policy MMC snap-in, under Local Policies -> User Rights Assignment). Without that privilege the SQL Server Agent service will not be able to impersonate the account to run the job step. Also it is important to note that proxy accounts do not automatically have access to your SQL Server. If, for example, you want to use a CmdExec or PowerShell job step to log back in to SQL Server, the proxy account must explicitly be granted a login back to your SQL Server (or otherwise inherit access from a Windows group, etc.).

http://blogs.msdn.com/b/uksharepoint/archive/2009/12/18/configuring-business-connectivity-services-bcs-search-in-sharepoint-2010.aspx

http://blogs.msdn.com/b/sharepointdev/archive/2011/07/26/rapid-application-development-using-infopath-and-bcs-part-2-anweshi-deverasetty.aspx

http://blogs.msdn.com/b/sharepointdev/archive/2011/07/28/rapid-application-development-using-infopath-and-bcs-part-3-anweshi-deverasetty.aspx

Create the XML for a BDC model.

http://msdn.microsoft.com/en-us/library/ee559278.aspx

BDC XML snippets to use when editing a file.

http://msdn.microsoft.com/en-us/library/gg607166.aspx#AdvancedAssociationsInBCS_ImplicitForeignKey

ALTER AdventureWorks database and edit the Exported XML for the model to create an association without a declarative FK.

http://msdn.microsoft.com/en-us/library/ee558413(office.14).aspx

Business Connectivity Services Schema Reference