Wednesday, November 5, 2014

mongo DB

MongoDB is a document database that provides high performance, high availability, and easy scalability.

Document Database

  • Documents (objects) map nicely to programming language data types.
  • Embedded documents and arrays reduce need for joins.
  • Dynamic schema makes polymorphism easier.
High Performance
  • Embedding makes reads and writes fast.
  • Indexes can include keys from embedded documents and arrays.
  • Optional streaming writes (no acknowledgments).
High Availability
  • Replicated servers with automatic master failover.
Easy Scalability
  • Automatic sharding distributes collection data across machines.
  • Eventually-consistent reads can be distributed over replicated servers.
Advanced Operations

MongoDB Data Model

A MongoDB deployment hosts a number of databases. A manual:database holds a set of collections. Amanual:collection holds a set of documents. A manual:document is a set of key-value pairs. Documents have dynamic schema. Dynamic schema means that documents in the same collection do not need to have the same set of fields or structure, and common fields in a collection’s documents may hold different types of data.

MongoDB Queries

Queries in MongoDB provides a set of operators to define how the find() method selects documents from a collection based on a query specification document that uses a combination of exact equality matches and conditionals using a query operator.

Deployment Architectures

Although MongoDB supports a “standalone” or single-instance operation, production MongoDB deployments are distributed by default. Replica sets provide high performance replication with automated failover, while sharded clusters make it possible to partition large data sets over many machines transparently to the users. MongoDB users combine replica sets and sharded clusters to provide high levels redundancy for large data sets transparently for applications.

MongoDB Design Philosophy


MongoDB wasn’t designed in a lab. We built MongoDB from our own experiences building large scale, high availability, robust systems. We didn’t start from scratch, we really tried to figure out what was broken, and tackle that. So the way I think about MongoDB is that if you take MySql, and change the data model from relational to document based, you get a lot of great features: embedded docs for speed, manageability, agile development with schema-less databases, easier horizontal scalability because joins aren’t as important. There are lots of things that work great in relational databases: indexes, dynamic queries and updates to name a few, and we haven’t changed much there. For example, the way you design your indexes in MongoDB should be exactly the way you do it in MySql or Oracle, you just have the option of indexing an embedded field.

—Eliot Horowitz, MongoDB CTO and Co-founder
  • New database technologies are needed to facilitate horizontal scaling of the data layer, easier development, and the ability to store order(s) of magnitude more data than was used in the past.
  • A non-relational approach is the best path to database solutions which scale horizontally to many machines.
  • It is unacceptable if these new technologies make writing applications harder. Writing code should be faster, easier, and more agile.
  • The document data model (JSON/BSON) is easy to code to, easy to manage(dynamic schema), and yields excellent performance by grouping relevant data together internally.
  • It is important to keep deep functionality to keep programming fast and simple. While some things must be left out, keep as much as possible – for example secondaries indexes, unique key constraints, atomic operations, multi-document updates.
  • Database technology should run anywhere, being available both for running on your own servers or VMs, and also as a cloud pay-for-what-you-use service.

Key MongoDB Features

MongoDB focuses on flexibility, power, speed, and ease of use:

Flexibility

MongoDB stores data in JSON documents (which we serialize to BSON). JSON provides a rich data model that seamlessly maps to native programming language types, and the dynamic schema makes it easier to evolve your data model than with a system with enforced schemas such as a RDBMS.

Power

MongoDB provides a lot of the features of a traditional RDBMS such as secondary indexes, dynamic queries, sorting, rich updates, upserts (update if document exists, insert if it doesn’t), and easy aggregation. This gives you the breadth of functionality that you are used to from an RDBMS, with the flexibility and scaling capability that the non-relational model allows.
Speed/Scaling

By keeping related data together in documents, queries can be much faster than in a relational database where related data is separated into multiple tables and then needs to be joined later. MongoDB also makes it easy to scale out your database. Autosharding allows you to scale your cluster linearly by adding more machines. It is possible to increase capacity without any downtime, which is very important on the web when load can increase suddenly and bringing down the website for extended maintenance can cost your business large amounts of revenue.

Ease of use

MongoDB works hard to be very easy to install, configure, maintain, and use. To this end, MongoDB provides few configuration options, and instead tries to automatically do the “right thing” whenever possible. This means that MongoDB works right out of the box, and you can dive right into developing your application, instead of spending a lot of time fine-tuning obscure database configurations.

Operations


MongoDB is a server process that runs on Linux, Windows and OS X. It can be run both as a 32 or 64-bit application. We recommend running in 64-bit mode, since MongoDB is limited to a total data size of about 2GB for all databases in 32-bit mode.

The MongoDB process listens on port 27017 by default (note that this can be set at start time - please see mongod options for more information).

Clients connect to the MongoDB process, optionally authenticate themselves if security is turned on, and perform a sequence of actions, such as inserts, queries and updates.

MongoDB stores its data in files (default location is /data/db/), and uses memory mapped files for data management for efficiency.

MongoDB can also be configured for data replication.

Additionally the MongoDB Management Service (MMS) application for managing MongoDB clusters using a simple user interface. MMS provides backup and monitoring. MMS is available to all users in the cloud and on-premises as part of MongoDB Standard and Enterprise Subscriptions.Agile and Scalable

MongoDB (from "humongous") is an open-source document database, and the leading NoSQL database. Written in C++, MongoDB features:
Document-Oriented Storage »

JSON-style documents with dynamic schemas offer simplicity and power.
Full Index Support »

Index on any attribute, just like you're used to.
Replication & High Availability »

Mirror across LANs and WANs for scale and peace of mind.
Auto-Sharding »

Scale horizontally without compromising functionality.
Querying »

Rich, document-based queries.
Fast In-Place Updates »

Atomic modifiers for contention-free performance.
Map/Reduce »

Flexible aggregation and data processing.
GridFS »

Store files of any size without complicating your stack.
MongoDB Management Service »

Manage MongoDB on the cloud infrastructure of your choice.
MongoDB Enterprise »

The best way to run MongoDB in production. Secured. Supported. Certified.
Production Support »

Our experts at your fingertips. Get access to our global support organization 24x365

Wednesday, October 29, 2014

Oracle Goldengate Technology

After Oracle corp. acquiring Goldengate software there is a lot of buzz about Oracle Goldengate and it is one of the hot topics at Oracle open world 2010.Oracle Goldengate can be used as a replication tool, ETL, and even as a DR solution.

Oracle Goldengate (Golden Gate) is probably the best replication software and it is very easy to configure and deploy it in large scale environment. Here are some of the things you need to be aware of:
  •  All Golden Gate configuration files are ascii text based files. Very easy to make changes but it is prone to human errors in an environment having many DBA's working on it.
  •  In order to use parallel apply threads, Golden Gate breaks down the database transaction into multiple transactions based on the hashing key defined for range split of the data. So, transactional consistency will not be guaranteed during real time but there won't be any data loss, but make sure that your application can tolerate this.
  • If there is no primary key or unique index exists on any table, Golden Gate will use all the columns as supplemental logging key pair for both extracts and replicats. But if you define key columns in the Golden Gate extract parameter file and if you don't have the supplemental logging enabled on that key columns combination, then Golden Gate will assume missing key columns record data as "NULL", which is a huge deal, and this will introduce logical data corruption on the target.
  •  Golden Gate started supporting bulk data loads with their 11.1 release but any NOLOGolden GateING data changes will be silently ignored without any warning.
  •  Golden Gate doesn't support compression on the source database.
  •  Golden Gate does support DDL replication but it is not easy to do selective DDL replication, it replicates every DDL that happens on the source database which is not desirable for some customers.
  • Tables being replicated to on the target can also be written to by any other application or DBA's.
  • Golden Gate supports ignoring data conflicts for updates after the first instantiation of the target database until it catches up. But it is very easy to forget turning off that parameter and any updates being lost will not be alerted by Golden Gate.
  • Golden Gate still works by reverse engineering the Oracle redolog. This may not be totally true with Golden Gate 11, but I expect Golden Gate to interpret Oracle redo more directly in later versions of 11 or 12.
  • Golden Gate dynamically decides to change the key columns that form the supplemental logging based on the state of primary key (i.e. in VALIDATED or NONVALIDATED state), which can introduce data corruptions on the target databases as the expected key columns data is missing in the trail files and they will be set to NULL. They now have the patch available for this, you can set "_USEALLKEYCOLUMNS and ALLOWNONVALIDATEDKEYS" parameters in GLOBALS file to get around this problem.