Wednesday, April 30, 2014

MongoDB Aggregation

Aggregations operations process data records and return computed results. Aggregation operations group values from multiple documents together, and can perform a variety of operations on the grouped data to return a single result. In sql count(*) and with group by is an equivalent of mongodb aggregation.

Creating new collections for MongoDB Aggregation

Here I’m inserting  documents(rows) into Student collection(Table)

db.Student.insert ({Student_Name:"Kalki",  Class: "2", Mark_Scored:100, Subject: ["Telugu", "English", "Maths"]})
db.Student.insert ({Student_Name:"Matsya", Class: "1", Mark_Scored:10,  Subject: ["Telugu", "English"]})

db.Student.insert ({Student_Name:"Krishna",Class: "1", Mark_Scored:50,  Subject: ["Telugu"]})
db.Student.insert ({Student_Name:"Buddha", Class: "2", Mark_Scored:60,  Subject: ["Telugu"]})
db.Student.insert ({Student_Name:"Rama",   Class: "2", Mark_Scored:80,  Subject: ["Telugu"]})

db.Student.insert ({Student_Name:"Krishna",Class: "1", Mark_Scored:50,  Subject: ["English"]})
db.Student.insert ({Student_Name:"Buddha", Class: "2", Mark_Scored:60,  Subject: ["English"]})
db.Student.insert ({Student_Name:"Rama",   Class: "2", Mark_Scored:80,  Subject: ["English"]})

db.Student.insert ({Student_Name:"Matsya", Class: "1", Mark_Scored:67,  Subject: ["Maths"]})
db.Student.insert ({Student_Name:"Krishna",Class: "1", Mark_Scored:95,  Subject: ["Maths"]})
db.Student.insert ({Student_Name:"Buddha", Class: "2", Mark_Scored:88,  Subject: ["Maths"]})
db.Student.insert ({Student_Name:"Rama",   Class: "2", Mark_Scored:40,  Subject: ["Maths"]})
Inserting and finding documents from collection

$sum:-
Sums up the defined value from all documents in the collection.
>db.Student.aggregate([{$group : {_id : "$item", num_tutorial : {$sum : "$Mark_Scored" }}}])

Ex: Lets try and get the sum of all the marks scored by each and every student, in Class "2"
db.Student.aggregate ([
   {
      "$match":
      {
         "Class": "2"
      }
   },
   {
      "$unwind": "$Subject"
   },
   {
      "$group":
      {
         "_id":
         {
            "Student_Name" : "$Student_Name"
         },
         "Total_Marks":
         {
            "$sum": "$Mark_Scored"
         }
      }
   }
])



$avg:-
> db.Student.aggregate([{$group : {_id : "$item", num_tutorial : {$avg : "$Mark_Scored" }}}])

Ex:
db.Student.aggregate ([
   {
      "$match":
      {
         "Class": "2"
      }
   },
   {
      "$unwind": "$Subject"
   },
   {
      "$group":
      {
         "_id":
         {
            "Student_Name" : "$Student_Name"
         },
         "Total_Marks":
         {
            "$sum": "$Mark_Scored"
         }
      }
   }
])



$match

Example:We want to consider only the marks of the students who study in Class "2"

db.Student.aggregate ([
  {
     "$match":
     {
        "Class":"2"
     }
  }
  ])

And results is




Let us say, we want to consider only the marks of the students who study in Class "2" and whose marks are more than or equal to 80
db.Student.aggregate ([
  {
     "$match":
     {
        "Class":"2",
        "Mark_Scored":
        {
           "$gte": 80
        }
     }
  }
  ])
And results is

$unwind


This will be very useful when the data is stored as list. When the unwind operator is applied on a list data field, it will generate a new record for each and every element of the list data field on which unwind is applied. It basically flattens the data. Lets see an example to understand this better

Note: The field name, on which unwind is applied, should be prefixed with $ (dollar sign)

Example:Lets apply unwind over "Kalki"'s data.
db.Student.aggregate ([
   {
      "$match":
      {
         "Student_Name": "Kalki",
      }
   }
])

Storing aggregate values into another collection:-
db.newStudent.insert( db.Student.aggregate ([
  {
     "$match":
     {
        "Class":"2"
     }
  }
  ]) )



Wednesday, April 23, 2014

Microsoft products that support SQL Server Always On.


The following is a list of Microsoft Products that support SQL Server AlwaysOn Availability Groups:

 
Product
Supported
Reference
Dynamics AX 2009 (Dynamics AX 2009 SP1)
No
Dynamics AX 2012
Yes
Dynamics CRM 4.0
No
Dynamics CRM 2011
Yes
Dynamics CRM 2013
Yes
Lync (2013 and earlier versions)
No
SharePoint Server 2010
Yes
SharePoint Server 2013
Yes
System Center 2012 Service Pack 1 (SP1) App Controller
Yes
System Center 2012 Service Pack 1 (SP1) Configuration Manager (SCCM)
No
System Center 2012 Service Pack 1 (SP1) Data Protection Manager (DPM)
No
System Center 2012 Service Pack 1 (SP1) Orchestrator
Yes
System Center 2012 Service Pack 1 (SP1) Operations Manager (SCOM)
Yes
System Center 2012 Service Pack 1 (SP1) Service Manager
Yes
System Center 2012 Service Pack 1 (SP1) Virtual Machine Manager (VMM)
Yes
System Center 2012 R2 App Controller
Yes
System Center 2012 R2 Configuration Manager (SCCM)
No
System Center 2012 R2 Data Protection Manager (DPM)
No
System Center 2012 R2 Orchestrator
Yes
System Center 2012 R2 Operations Manager (SCOM)
Yes
System Center 2012 R2 Service Manager
Yes
System Center 2012 R2 Virtual Machine Manager (VMM)
Yes
Team Foundation Server 2010
No
Team Foundation Server 2012
Yes
Team Foundation Server 2013
Yes
Windows Server 2012 - Cluster Aware Updating (CAU)
NO


Friday, April 18, 2014

Save Thousands in Licensing Costs for SQL Server AlwaysOn Availability Groups


There are a number of key reasons NOT to use Windows Server 2008 R2 for AlwaysOn Availability Groups (AGs), including:
·         It’s an older operating system—released nearly 4.5 years ago. A lot has changed in 4.5 years. Windows Server 2012 and 2012 R2 may come with the metro interface—which is a bit stupid to put on a server—but they’re much faster, more stable, and more secure operation systems by a wide margin and also picks up vastly improved SMB (as an example of some of the improvements you can expect).

·         Windows Server Failover Clustering (WSFC) wasn’t mature enough to handle AGs. Availability Groups may not be as hard to set up and configure as AlwaysOn Failover Cluster Instances (FCIs) or traditional clusters, but they’re still based upon WSFC. As you know, WSFC on SQL Server 2008 R2 simply wasn’t mature enough to tackle some of the use cases encountered by AGs—hence some of the initial horror stories about spectacular AG failures on Windows Server 2008 R2, along with the need to track down and install gobs of hard-to-find hot-fixes/patches on Windows Server 2008 R2 before CORRECTLY and SAFELY deploying Availability Groups.

·         It is, in most cases, MORE EXPENSIVE to license. Failover Clustering on Windows Server 2008 R2 was only available as part of the Enterprise Edition (or higher) SKUs—much more expensive to license than Windows Server Standard Edition. With Windows Server 2012 and above, Standard Edition now picks up FULL support for Clustering (not just simple 2-node active/passive Clusters only—but full-blown Clustering support).
Windows Server 2012 R2 DataCenter Edition picks up a roughly 28 percent price increase, but Standard Edition stays roughly the same price. The only caveat, of course, is that Windows Server 2012 and above, need to be licensed via 2-Processor Packs—which is a pain for smaller servers—and that CALs are no longer included in most pricing options as part of the license for Windows Server 2012 and above. Or, in short, licensing Windows is always complex, convoluted, and cryptic enough that you typically need to spend a solid 20 minutes perusing documentation before you can actually feel that you get a semblance of a grip on what you need to buy/license to provision new servers.
WS 2012 R2 Standard Edition is Great for Availability Groups
The point I’m trying to get across, though, is that in most environments, Windows Server 2012/2012 R2 Standard Edition is enough to keep SQL Server AlwaysOn Availability Groups adequately and amply powered and happy. In fact, the only differentiator between the DataCenter Edition and the Standard Edition is that DataCenter Edition provides unlimited hosting of virtual machines, or VMs (i.e., license the physical processors and run as many VMs as your hardware can handle) whereas, Standard Edition allows only 2 VMs per License 2-Pack (with any further VMs requiring their own, actual, licenses), and is really geared more towards traditional or physical workloads—which is what you're going to want in most cases for your AGs, anyhow. 
Other than that, BOTH versions of Windows Server support the same amount of hardware, throughput, and are full-featured versions of Windows Server (i.e., they’re not artificially restricted in any way).
Drastic Difference in Pricing
The difference in pricing, though, is pretty substantial. Not including CALS, Windows Server 2012 R2 Standard Edition typically runs around $882 for 2 processors, while Windows Server 2012 R2 DataCenter Edition typically costs around $6,100 for 2 processors (Windows Server 2012 DataCenter Edition cost roughly 28 percent cheaper at around $4,800 for 2 processors).
So, in short, Windows Server 2012 or Windows Server 2012 R2 Standard Edition is a great way to save thousands on licensing—and save yourself some serious headaches when deploying AlwaysOn Availabiliity Groups. 


Thursday, April 17, 2014

SQL Server 2014 Key Features and Enhancements

SQL Server 2014 Key Features and Enhancements
By Lakshmi Narayana Reddy.L
SQL Server 2014 comes with a set of enhancements as compared to its processors that brings a lot of business benefits to the applications in OLTP and OLAP environments. This article highlights a few key enhancements that are built into the product.
Performance Enhancements
These are performance enhancements that can help your workload run faster.
·         In-Memory OLTP:  This is a new feature allows the database to handle in-memory operations in OLTP scenarios (resolve issues in high concurrency situations). This component is called as “Hekaton”.  Microsoft has released an in-memory component called ‘xVelocity’catering to OALP requirements, along with SQL Server 2012.
·         On-line indexing at the partition level: This feature allows index rebuilding done at a partition level. Also, the index statistics can be managed at the partition level which will be a huge improvement in performance
·         Updatable Column Store Indexes (CSI):  The CSI feature has been introduced in 2012. The limitation with CSI was that the table cannot be modified once a CSI is created.  In order to update table information the index needed to be dropped or disabled and then rebuilt the index. This new feature provides the ability to load or delete data from the table with Column Store indexes.
·         Buffer Pool Extension to Solid State Drives (SSDs). Ability of each node to have its own SSD or SSD Array for buffering (just like you would with TempDB) and thus increase in the performance by means of faster paging.  In this way, one can cache frequently used data on SSDs. This feature can be best leveraged in case of read-heavy OLTP workloads. 
·         Resource Management. Resource Governor can control  I/O along with CPU and Memory (provided by the previous versions).
Improved Scalability
SQL Server has imcreased the amount of hardware it can use.
·         Ability to scale up to 640 logical processors and 4TB of memory in a physical environment
·         Ability to scale to 64 virtual processors and 1TB of memory when running in a virtual machine (VM).
High-Availability Enhancements
·         AlwaysOn Availability Groups (AG) get more secondary nodes: Always On supports now up to 8 secondary nodes instead of 4 (is the case with SQL 2012). Of course, Enterprise Edition is needed.
·         AlwaysOn Availability Group’s Readable Secondary will be ON-LINE (more reliable). In SQL 2012, if the primary drops offline, the readable replica databases drop offline. In SQL 2014, the secondary remain online and readable when the primaries aren’t available.
·         Azure Integrated AlwaysOn Availability Groups:  Uses Azure VMs as AlwaysOn AG replicas. This replicas can be createdasynchronously on cloud (Azure platform) that saves from paying for expensive offsite datacenter space with machines that sit idle all the time.  
Backup Enhancements
·         Smart Backup to Azure (Windows Azure Integrated Backup): Another new backup feature is Smart Backups. With Smart Backups SQL Server determines whether a full or incremental backup is needed and backs up accordingly to Azure.
·          Azure backup feature and the Azure AlwaysOn Availability options are completely integrated into SSMS.
Microsoft® in-memory database engine
Microsoft® implemented an in-memory transactional engine with the project code name Hekaton*”. Hekaton is expected to dramatically improve the throughput and latency of SQL Server’s on-line transaction processing (OLTP) capabilities. Hekaton is designed to meet the requirements of the most demanding OLTP applications for financial services companies, online gaming and other companies which have extremely demanding TP requirements. This product achieves breakthrough improvement in TP capabilities without requiring a separate data management product or a new programming model. It’s still SQL Server!
Note*: Hekaton is from the Greek word ?κατ?ν for “hundred”. The design goal for the Hekaton original proof of concept prototype was to achieve 100x speeds (possibly)  for certain TP operations.
Key Features:
·         Implements a row-based technology squarely focused on transaction processing (TP) workloads. However, the xVelocity* and Hekaton in-memory approaches are NOT mutually exclusive. The combination of Hekaton and SQL Server’s existing xVelocity column store index and xVelocity analytics engine, will result in a great combination.
·         Hekaton (In-memory TP engine) and xVelocity column store index will be built-in to SQL Server, rather than a separate data engine, which is a conscious design choice
Note*: xVelocity is OLAP version of in-memory database released along with SQL Server 2012.
Technology Implementation:
·         Hekaton works by providing in-application memory storage for the most often used tables in SQL Server. Identifies tables that are most accessed, and will store them in the system’s main memory for faster access time.
·         Hekaton compiles T-SQL stored procedures directly into native code for faster execution.
·         Hekaton uses a new concurrency control mechanism developed by Microsoft® team and researchers from the University of Wisconsin using lock-free data structures for better scalability across multiple cores, avoiding locks while preserving ACID transaction integrity.
These features may be available already outside SQL Server such as own Power Pivot and Power View. However, the biggest difference is that Hekaton is built directly into SQL Server, so that there are no extensions, downloads, or interfaces that can slow down the very program meant to help increase your speed.
A few challenges with the implementation of in-memory OLTP database are as follows.
·         Need to change your data model. Need to bring significant changes to the traditional OLTP model. For ex: Identity fields aren’t supported, may have to use a GUID as a primary clustered key.
·         Need to change application code to replace ad-hoc SQL queries with stored procedure calls. Hekaton works best with stored procedures, as the stored procedures can compile into native code.
·         Since the processing happens in in-memory only, if there is any sudden growth in the Hekaton tables, we can cache less of your other tables. We may run out of memory.