Magic Dragon Books - Aviation, military and history

Besides IT, military aviation and history is a big part of my life and reading materials since I was in elementary school. In the past years, I wrote and translated articles either in Chinese or English through various publishing medias, digital or printed. I also get to know other people who have the same interest and expertise and become good friends.

Some of them are specialized in the Republic of China (ROC) air force history, either during the war with Japan or after 1945 with Communist China, including secret reconn missions which only been published or heard in recent years.

Magic Dragon Books is an on line book store to bring you these military aviation history unknown to most of the world in Chinese or English. I am not part of the company or business operation, neither details of the company itself. I know this due to a respectable and long time friend who dedicated himself in this field for years.

On the other hand, I always believe history has multiple faces and we can only understand more from various sources. Some of the ROC air force operations involved missions with PLA. Compare to the influence red China has to the world, it it more important than ever to bring up the stories from the other side, not to correct anything, but to provide a different angle, data sources or point of view.

If you are interested to know more about the published materials, please visit the site at: http://magicdragonbooks.com/

Find user permissions assigned in one or more site collections of all items

MOSS does not provide out-of-box interface to view assigned permissions to the same user cross site collections. Although there are tools to help you, no matter it's paid or free, if your environment does not allow free tools and you have to find it for now, what can you do?

Since everything is stored in SQL database, why not write some SQL statements to find them? That's what I have been doing before we have a nice tool to use. If you already know the structure of tables used by MOSS, then you may be able to do it in a short period of time.

If not, here is the SQL query you can use to do it:

Select 'http://yourportal/'+a.scopeurl as ObjectURL, a.title as UserID,roles.title as Permission,a.SPgroup, a.ADLogin
from roles inner join
(
select perms.scopeid, perms.scopeurl, perms.roledefwebid, alluser.title, alluser.ADLogin,alluser.SPgroup,roleassignment.roleid, roleassignment.siteid
from perms inner join
roleassignment on
perms.siteid=roleassignment.siteid
and
perms.scopeid=roleassignment.scopeid
inner join(
(
select groups.siteid, groups.id as id,userinfo.tp_login as ADLogin,userinfo.tp_title as title, groups.title as SPGroup
from groups inner join
groupmembership on groups.siteid=groupmembership.siteid
and
groups.id=groupmembership.groupid
inner join userinfo on
groupmembership.siteid=userinfo.tp_siteid
and
groupmembership.memberid=userinfo.tp_id)

union

select tp_siteid as siteid, tp_id as id,userinfo.tp_login as ADLogin, tp_title as title, 'no' as SPGroup from userinfo) as Alluser
on
roleassignment.siteid=alluser.siteid
and
roleassignment.principalid=alluser.id) as a
on
roles.siteid=a.siteid
and
roles.webid=a.roledefwebid
and
roles.roleid=a.roleid

Let me explain the columns you get:

1. frist colunmn is the whole URL of an object (site, , sub site, library/list, folder oritem) which has unique permission
2. The User ID
3. Permission assigned to this user.
4. SharePoint User Group this user is assigned to. If the permission assigned to this user individually, this column shows No.
5. User log in ID.

So, this query will tell you in a single content database, what are those objects have unique permissions, who has permission, what type permission assigned, and this permission assigned to a user group or individually.

Enjoy it.

SharePoint 2010 Preview

Microsoft released three SharePoint 2010 preview videos: Overview, For IT Professional and For Developer. You can find them here.

Work Around STSADM Import/Export Failure

We have been using STSADM Import/Export features to reorganize the entire MOSS content structure. Sometimes we saw fatal or too many error during importing process, especially when we worked on site collections with large number or size of contents.

After couple tries, we found that if we choose not to compress the exported files, it really helped to draw down the chance to have errors.

If you experienced similar situation, try this work around and let me know if it really helps you.


Enjoy.

MOSS and Reporting Services Integrated with NTLM: is it possible?

Can you integrated SQL Services Reporting Services into your MOSS farm with NTLM only?

If you search around, lots of articles do not tell you exactly which one, NTLM or Kerberos, to use. Others may tell you that you have to use Kerberos and it means you have to go through lots of troubles to just get Kerberos done. Of course, if you have someone every good with this and can help you without waiting tooo long, then you do not have to read the rest of this article.

I have been looking for this information for a year and half and finally I have the answer recently: Yes, you can use NTLM to integrate these two application together.

There are two steps never documented in any MOSS and SSRS integration information and a large portion of them just assumed you have Kerberos in place.

First, the SSRS web site has to use NTLM authentication in IIS. You can check this KB article to find out whether SSRS is using mixing or NTLM. Run the cscript adsutil.vbs set command to set it to NTLM.

Second, in the Manage integration settings page of Central Administration site, you can only use Trusted instead of Windows authentication.

With these undocumented settings, we are able to design and run reports from a MOSS site now.

Hopefully this information is able to help you if it's not too late.

Best Practice Conference August 24-26, 2009

Best Pracice Conference is back to Washignton DC this Aughst. This time it's for SharePoint and SQL Server.

You can find details here.

MOSS and High CPU Usage - a rare case

Earlier this week in the afternoon we noticed that the production MOSS was running very slow. Our team found out the CPU usage of the WFE server was reaching 100% constantly. The WFE had 8 cores with 4 GB memory and hard to believe it's normal.

I was on the way home and driving, so I could only help with my mouth bfore I could open my computer and connect to the server. We reset IIS couple times and the same situation would come back after 10 to 15 minutes. We could not reboot at the time due to the users from the west still using it.

So the next thing we did was to find out what may cause this CPU spike. When we traced the time line back, the major difference was that one site collection was restored with DocAve earlier in the afternoon. We decided to narrow down the range by restoring the same site collection to our development environment.

Then the same situation happened in the development sever and we knew that's the source. But why? By going through some emails and conversation with different users, we thought it has something to do with InfoPath form. But we still did not have anything clear.

In order to put production MOSS back to normal status, we communicated to the site owners with the situation and our work around solutions: backed up the site collection and removed it from the system until we know how to open it again without problem. Very soon we had approval from the owner and proceeded.

We backed up the site collection, removed it from production, reset IIS and restarted coupld services. Everything looked fine and CPU usage staied low for 30 to 40 minutes. We knew the crisis in production was over and we can concerated on finding the answers.

Our techinical lead took another site collection which had InfoPath from and restored it with DocAve to development enevironment. The same situation came back right away. Now we knew something wrong with DocAve and InfoPath form.

We sent our finding and a sample to the vendor and they could reproduce the same problem in their lab. So, DocAve can not deal with site which has InfoPath form?

Not really. This is a very rare case: in order to have this problem, you have to have a InfoPath 2003 form template saved in a library which was migrated from WSS 2 (not sure the same would happen to 2003 Portal). When you restore the site with DocAve, you will have the same problem. However, if you restore the entire site collection with STSADM, no problem at all.

We received a patch from the vendor and tested it. So far it looks good. So, case closed.

How to Find Which Document Library Has Versioning Control

This was the question I got from my project manager and project lead. They wanted to know which document library had versioning control.

I thought this should be an easy task, 15 minutes in the databases and I should be able to find all of them.

Wrong. It took me 2 and half hours and I still could not tell exactly which library had this feature turned on or not.

What I found out was the Version column in the AllLists table had nothing to do with the versioing control. It represented how many times the settings of this list was changed.

I did some test and found out another column called tp_flag should have the information. When I changed from None to Major Only, to Major and Minor and then back to None. There were only three values shown under each setting. If I got these three integers, I should be able to query the entire table and identified all of them, right?

I was wrong again. The next thing I found out was: there were other group of integers used by different lists.For example, list A used 123, 124 and 125 .List B used 3456, 3789 and 4789 and so on. Not all lists were different and there should be some rules how the numbers were constructed.

However I haven't had time to go back and figure it out. If you have any information, I am more than happy to listen.

Thanks.

STSADM Export/Import Problems

In the past 1 1/2 month, our team is working on to re-organize the portal site structure. Before we migrated last December, there were more than 150 site collections and more than 700 sub sites. In our plan, we are going to put them into one portal and 9 departmental site collections. From 150 plus to 10 site collections, I believe this is a big efforts to move 85 GB database around multiple databases.

The first tool we consider is, of course, STSADM. This is the out-of-box tool from Microsoft, support by Microsoft and what can go wrong, right?

Actually, lots things can go wrong and so far I haven't found lots information from on line sources. So far the problems we have seen include:

1. Not all document versions are imported correctly. I did some research through database and so far I have one clue. There is a column called DocFlag in both AllDocs and AllDocVersions tables. If any record in AlldocVersions has a different DocFlag values than the current version, import process skipped this one.
2. Sometimes all of the versions are replaced by the current one, which means the date and author are also replaced.
3. Import operation can not process the Project Tracking list properly and I have no clue.
4. Sometimes not all sub sites, lists, libraries or items are not imported and I have no clue how to predicted. If I redo import or export/import, I can get everything correctly.
5. The import log does not show you all information. For example, you can find each version of a document is exported from the log, the same information is missing from the import log.
6. Do I mention all of these problems are hard to predicted? I have no idea if I wrong export/import to the same site collection multiple time, when do I get and which problems.
7. The biggest one and also no way to predict it: sometime when you import from an site collection to a group of sub sites, the original top-level site does not import to the correct sub site. The whole contents go to the top-level site of the new URL and overwrite the one. I have no clue or information why and how it happened.

After very intensive testing and evaluation, we decided not to use STSADM as the tool to do this re-organization. One major concern is we have no way to predict when and where the problems will strike us.

If you have any way to resolve the problems I just mentioned, I am all ears. Thanks in advanced.

Where to find the thrid site collection administrator?

When either the server or site collection administrator (depends on you have self-service turn on or not) creates a new site collection, the administrator has two field to enter at least the primary site collection administrator. This information is saved in the OwnerID and SecondaryContactID.

After the site collection is created, the site collection administrator is able to remove, change or add more than two site collection administrators to the list. If you remove a name from the list, MOSS will remove it from one of the column. When you add the same name back, or add the 2nd administrator to the list, you will not see the column updated by the system.

On the other hand, since there are just two columns, where the setting is saved if you add another administrator?

This is actually what happened when you make changes to the site collection administrator list:

1. When you add up to two names to the fields during site collection creation, both names are saved in the Sites table.
2. When you remove anyone from the list after the site collection is created, MOSS will remove it from the Sites table.
3. If you add a new name, no matter this is the 2nd , 3rd or more, MOSS will not update the Sites table.
4. All the changes made to the site collection administrator list go to UserInfo table. There is a column called SiteAdmin and this is the place you can do your search.

DocAve 5 needs more and better QA

We are using DocAve 4 to backup WSS 2 environment for about a year. It's not perfect and the interface sometimes corky, according to one of my team member, but it does the backup and restore jobs well.

We migrated from WSS 2 to MOSS 2007 early last month and star using DocAve 5 as our main disaster and recovery solutions and to our surprise, there are bugs or features do not look hard to identify and fix. Especially DocAve 5 has been out there for more than a year.

For example, in the Content Manager screen when you want to move site or site collections from one URL to another, pretty much like using STSADM Export and Import commands. In the tree view, there is a green icon next to the designated site and you can change it to point up, down or level. What is this for? We did not find any explanation in the user guide. When we had conference with AvePoint, there own people does not have good idea how and when to use it.

When we tried to move site collection during the conference call with their support, the end results looked pretty bad and they admitted they had to go back to do more testing.

Another example is also related to move site to new URL. If use DocAve's feature, any inactive user accounts do not import to the new site with the contents and they name changed to System Account. This looks like the using STSADM to do exporting and importing without using the -includeusersecurity parameter. However, I can choose to include user security or not but can not find any place in DocAve to choose it.

AvePoint's response was this is by design and it's not a bug. But they will consider to add a check box or something to provide different options.

If you use DocAve before, you know in the job status screen, the bar to show the progress may stop at certain percentage for while, then moves very fast to another number and stops again. In this version, it still acts like this and we can see it stops at 99% for a long time. So the progress bar does not give the users the real progress since from 99% to finish takes longer than moving from 5% to 71%. Then why not making it more realistic to the real progress? Especially this is pretty much the same behavior from the previous version.

The user guide is huge with lots of do this, click here type of instruction. However, for a user does not use DocAve before, it does not tell me why and when to use a feature or a particular option. List all commands and features do not give users a good idea of what they should do with different environments and circumstances.

I think the biggest surprise we had is: DocAve can not performance incremental or differential backup well. When we tested it in a development environment with migrated data, the daily differential backup process generated backed up file size around 40 to 50% of a full backup. However, it was a development and only 5 people to use it and not much changes daily. After couple long talks, AvePoint gave us a hot fix to deal with this problem. They said that because we had migrated sites and some of the URLs did not match the new server farm's URL and DocAve can not handle those contents.

Are we the first and few users with migrated contents and new URLs in MOSS 2007? I don't think so. Then how come this issue never raised any attention?

Anyway, I know DocAve has pretty good reputation to the SharePoint backup and restore solutions. However, this new version also wants to provide migration and administration functions which would be very helpful.. However, we felt that the product needs better and more quality control plus more information to how, why and when to use them.