Posted on

NoSQL vs SQL: Understand the Differences and Make the Best Choice

nosql vs sql
NoSQL vs SQL
The tough choice of NoSQL vs SQL. An important decision for any business that may pay the price for down the road if you don’t plan accordingly. Here, we break down the important distinctions and discuss NoSQL vs SQL for you to make an informed decision.

What is NoSQL

NoSQL or Not Only SQL is an alternative to relational databases such as MySQL, Postgres, and SQL Server. Typically NoSQL databases are popular for working with large sets of data as data is stored in the form of flat collections so reading or writing operations is much faster. There is also less management for NoSQL databases as they support automatic repair, data distribution, and simpler data models. MongoDB, Couchbase, and Cassandra are popular NoSQL databases, but what do their queries look like? If NoSQL is all about leaving SQL behind, then what happens? These are fair questions, for example MongoDB uses its own syntax that kind of resembles JavaScript. Let’s take a look: 
db.inventory.find( { status: "D" } )
In the above query the example selects from the inventory collection all documents where the status equals “D”. It is not too difficult to follow, but does feel a little foreign if you are used to SQL.

What is SQL

SQL or Structured Query Language is a language used in programming and designed for managing data held in a relational database management system. Some examples of popular SQL databases are Microsoft SQL Server, MySQL, and Postgres. One of many reasons for its popularity is how portable it is. SQL can be used in the program in PCs, servers, laptops, and even some of the mobile phones. Unlike NoSQL databases, SQL databases have an established, well defined standard. The language is also very straightforward, let’s take a look at it: 
SELECT * FROM inventory WHERE status = "D"
In the above query the example selects from the inventory collection all documents where the status equals “D”. Sound familiar? That is because it is doing the same thing the MongoDB query is doing.

Which is Better?

This all depends on what your goals are. SQL databases tend to be beneficial for companies that have a clear business structure and do not plan on making many changes to their database structure. If your company has rapid growth or no clear schema definitions then a NoSQL database is the right choice for you. If you’re uncertain about which type of database is best for your business, you can schedule a consultation with one of our experts today.

What if you want to use a SQL database but not learn SQL or the opposite, use a NoSQL database without learning how to handle the data? That is where DreamFactory comes into play, interact with your data without having to learn SQL. In minutes generate an API from nearly any datasource that is fully documented and secure without code. Contact an expert today to learn how to cut development cost and time in half.
Posted on

Rakuten RapidAPI and DreamFactory Partner Up On Open API Adoption

Rakuten RapidAPI and DreamFactory partner up on open API adoption We’re excited to announce that Rakuten RapidAPI, the world’s largest API marketplace is teaming up with DreamFactory, the fastest API creation and management platform. Our goal is to help our developer users streamline API development, usage and distribution. Rakuten RapidAPI and DreamFactory tools and services will allow developers to seamlessly create swagger format APIs, easily monetize and consume public APIs, and launch new services quickly and securely.

RapidAPI and DreamFactory

Today, Rakuten RapidAPI is the world’s largest API Marketplace with over 10,000 APIs. Our mission is to help developers and enterprises alike reduce engineering time, cost and effort through the power of public APIs. With such a comprehensive catalog, we’ve got something for any use case, all accessible from a single platform. We’ve also just hit our 1MN users milestone! For API Providers, this means access to a global user base on day 1. Our built in toolset lets you configure and set pricing for your API. We manage billing and collections, and pay out your share. DreamFactory API-as-a-Service offering helps startups to enterprises create Swagger 2.0 documented APIs with streamlined management features and military grade security. Today, DreamFactory hypercharges app development and digital transformation efforts for 400,000 developers- from startups to Global 2000 companies. Here’s what DreamFactory CEO James Crennan had to say about this partnership. “DreamFactory’s extensive open source community spans single developer startups to companies such as Nike and Walmart. A partnership with Rakuten RapidAPI helps our community build innovative products powered by best-in-class APIs. Similarly, we see Rakuten RapidAPI as a valuable channel for any of our clients are seeking to monetise their APIs.”

Why we’re teaming up

Rakuten RapidAPI and DreamFactory products are both built with standardised interfaces and integrations as in mind improve developer experience. As both the amount of software and their underlying complexity increase, developers have to manage multiple tools, services, and integrations. The Rakuten RapidAPI API Marketplace goes beyond aggregating APIs. Our standard API portal includes built-in tooling for API configuration, endpoint testing, service subscriptions, and code snippet in 11 programming languages. This means developers no longer have to juggle multiple accounts, navigate multitudes of developer portals, or monitor API performance from multiple dashboards. This makes us the only platform you need to find, test, and connect to public APIs.

RapidAPI and DreamFactory

Experts from Gartner and Forrester consistently point out that API creation is the single largest burden for development efforts. It’s hard to keep to a completion schedule, they’re subject to wildly different code standards, and introduce ‘key man risks’ through purposefully difficult code that stifles on-boarding. As a result, internally built APIs end up being minimally reused, resulting in duplicated code that requires ongoing maintenance. At the same time, APIs are vital to enabling modern, digital applications. That’s why DreamFactory set out to fix today’s broken API creation efforts. DreamFactory’s Swagger documentation provides a standardized URL structure to every database, meaning the way you call information through an API for SQL Server is the same as for MongoDB. DreamFactory also offers a standardized method of managing your portfolio of APIs from a central point – bringing sanity, control and giving back productivity. Here’s what their standard Swagger documentation looks like across any DB. Need a standardized API in a flash? Reach out to the DreamFactory team to find out how they can help you build it in less than 30 seconds.

About Rakuten RapidAPI



RapidAPI and DreamFactory

Rakuten RapidAPI is the world’s largest API marketplace with over 8,000 APIs used by 500,000 developers around the world. Our platform helps software developers and enterprise engineering teams seamlessly test and integrate open APIs for their applications, saving them previous engineering time, effort and cost. Companies like Microsoft, Sendgrid, and Crunchbase are already our valued partners.

Facebook | LinkedIn | Twitter

About DreamFactory



RapidAPI and DreamFactory

DreamFactory is the fastest API management solution on the market, allowing users to generate standardised, secure and fully documented APIs in under a minute. We satisfy an open source community of over 400,000 as well as serving commercial clients in Fortune 500 companies. Our platform allows developers to rapidly integrate any number of data sources through single or multiple APIs, saving development time and streamlining application development.
Posted on

Monitoring APIs with Grafana

DreamFactory, Prometheus, Docker, and Grafana
DreamFactory, Prometheus, Docker, and Grafana
Now that we have explored monitoring apis with Prometheus, lets take a look at monitoring our APIs with Grafana. You may have noticed from the previous blog that Prometheus is awesome, but takes some time to fully flesh out and their dashboards aren’t the best. Good thing Grafana specializes in data visualization. We will not even have to configure all our Prometheus queries, so let’s get started.

Monitoring APIs with Grafana

We first need to confirm everything is running, if so your output should look similar to the above image. If you are not familiar with getting everything up and running please reference our previous blog. By navigating to 127.0.0.1:3000 you should see the Grafana home page. If you are not familiar with Grafana you may be wondering where the dashboards are. They are all located in the top left dropdown where we will see 4 pre-configured dashboards. Let’s review each of them to understand what each of them does.

Docker Host Dashboard

The Docker Host Dashboard shows metrics for monitoring the usage of the server. Thanks to dockerprom(link github repo) for pre-configuring all this we can now see:
  • Server uptime, CPU idle percent, number of CPU cores, available memory, swap and storage.
  • System load average graph, running and blocked by IO processes graph, interrupts graph.
  • CPU usage graph by mode (guest, idle, iowait, irq, nice, softirq, steal, system, user).
  • Memory usage graph by distribution (used, free, buffers, cached).
  • IO usage graph (read Bps, read Bps and IO time).
  • Network usage graph by device (inbound Bps, Outbound Bps).
  • Swap usage and activity graphs.
Monitoring APIs with Grafana

Docker Containers Dashboard

The Docker Containers Dashboard shows key metrics for monitoring running containers:
  • Total containers CPU load, memory and storage usage.
  • Running containers graph, system load graph, IO usage graph.
  • Container CPU usage graph.
  • Memory usage graph.
  • Cached memory usage graph.
  • Network inbound usage graph.
  • Container network outbound usage graph.
Monitoring APIs with Grafana

Monitor Services Dashboard

The Monitor Services Dashboard shows key metrics for monitoring the containers that make up the monitoring stack:
  • Prometheus container uptime, monitoring stack total memory usage, Prometheus local storage memory chunks and series.
  • Container CPU usage graph.
  • Memory usage graph.
  • Prometheus chunks to persist and persistence urgency graphs.
  • Chunks ops and checkpoint duration graphs.
  • Samples ingested rate, target scrapes and scrape duration graphs.
  • Alerts graph.
Monitoring APIs with Grafana

Conclusion

We now have a fully functioning monitoring tool with no code. Isn’t that awesome? A no code API tool and a no code data visualization tool, it can’t get much better than this. This post has only scratched the surface with regards to what you can accomplish with these tools. If you are interested in this for a Ruby application, check out our friends over at Scout who wrote a blog about doing exactly that. If you like what you see, sign up for a free trial today or contact us for more information!
Posted on

Monitoring APIs with Prometheus

DreamFactory, Prometheus, Docker, and Grafana
DreamFactory, Prometheus, Docker, and Grafana
Have you ever wanted a quick and easy way to monitor your Docker environment? Do you want to have an API monitor with minimal configuration? Well it looks like there has never been a better time to monitor your API as there is a plethora of awesome tools out there to help us such as Prometheus and Grafana. If you would like to follow along with this blog start by downloading DreamFactory from our github repo, or if on-prem isn’t for you try our new hosted solution for free today!

Installing DreamFactory

To spin up your Docker instance fast start by cloning our Docker repository 
git clone https://github.com/dreamfactorysoftware/df-docker.git
cd df-docker
You will then want to build the image 
docker-compose build
Now that it is built lets start them up 
docker-compose up -d
Once you go to 127.0.0.1 in your browser, it may take some time, you will be asked to create your first admin user. For more installation options check out our README.

SQL Server API

Monitoring with Prometheus

If we navigate to http://127.0.0.1:9090, we will now see Prometheus. It is an open-source systems monitoring and alerting toolkit. To see what is happening, you have to run queries. To help give you an idea of what is happening, lets start with a basic query to see how many containers are running. 
scalar(count(container_memory_usage_bytes{image!=""}) > 0)
Shows we have 10 containers running and give us a little graph.

SQL Server API
An advantage of Prometheus is alerts. To get alerts we can run: 
sum(ALERTS{alertstate="firing"}) by (alertname)
This will return any active alerts that are firing letting us know if we need to take action. In my case Jenkins is down so Prometheus is able to alert me and if anything else were to go down it can be found here.

SQL Server API

There are plenty of other queries that can be ran and I highly encourage you to explore Prometheus to see its true potential. Check out Prometheus queries yourself and create your own. If you are interested in configuring this for a Ruby application, check out our friends over at Scout who wrote a blog about doing exactly that.

Conclusion

Now we have DreamFactory and Prometheus communicating. Prometheus dashboards are not that impressive for the time being, so in the next blog we will be exploring how to make Prometheus and Grafana work together to build the best dashboards. Check back often for updates and the next blog: Monitoring APIs with Grafana
Posted on

Creating a Microsoft SQL Server API in Less Than 5 minutes with DreamFactory

DreamFactory and Microsoft SQL Server
DreamFactory and Microsoft SQL Server
Do you have a ton of data sources and do not know how to expose them? Do you know you need a SQL Server API but don’t know where to begin to build it? Look no further, DreamFactory can take any database and generate a fully documented and secure REST API faster than making a sandwich. All you need is your database credentials and DreamFactory will handle the rest, instantly generating Swagger documentation and securing your API by way of API keys. Follow along with the blog or our video below!

Installing DreamFactory

In order to get started you must install DreamFactory, which is Open Source and gives you the ability to try out a numerous amount of popular databases. To spin up your own hosted environment for free click here.

Generating the SQL Server API

Now the fun part. Once you have DreamFactory up and running you will be on the Admin panel. From there you will navigate to the Services tab to connect your database for your API. For example I have selected a Microsoft SQL Server database but it is nearly identical for any database you will be connecting to. Connecting your database typically only requires filling out these 5 fields as shown below.

SQL Server API

Upon saving there will be a success window pop-up saying “Service saved successfully”. What it doesn’t tell you is all the magic it just did behind the scenes. In just that short amount of time, it generated your REST API. So now if you navigate to the API Docs tab you can see your new documentation for the API and actually interact with it via the “Try it out” button.

SQL Server API

Securing and Interacting with the API

I can end it here now that you have generated your API, but where is the fun in that? Now let’s actually see the API in action! First things first, let’s generate an API key to be paired with the API for security purposes. DreamFactory does not allow access to the API without being authenticated. Let’s navigate to the Roles tab and create a Role for our API. This Role with correspond with the API key so different users can have different privileges based off different keys.

SQL Server API

For my Role I have pointed it to the SQL Server Service we just created and told it to only allow GET calls on the endpoints. This will ensure anyone using this API key will not be able to, for example delete data from the database. We must now link this Role to an API key. If we now go to the Apps tab we can create a new API key with the corresponding Role.

SQL Server API

Once we hit save we are able to see the API key generated for our use. Just to show how it works I will be using Insomnia, a popular HTTP service, to call our DreamFactory API. I will call the customers table, passing the API key in the headers for authentication.

SQL Server API

Conclusion

As you can see I have access to our new API and how much time did that take? Way less time than building this API yourself! If you would like to find out exactly how much time and money DreamFactory can save you, check out our API calculator. Otherwise what are you waiting for? Go build your next application using DreamFactory already! If you have any questions about the platform, or just APIs in general, we’d love to hear from you! Contact us.
Posted on

Learn About DreamFactory and NoSQL – NoSQL Database Podcast Interview

DreamFactory CTO Jason Gilmore recently sat down with NoSQL Database Podcast host Matt Groves to talk about DreamFactory, NoSQL database solutions, and advanced API devops topics such as performance, profiling, and logging. Here’s a chronological breakdown of the conversation:

  • 00:48 – Introduction to Jason
  • 02:51 – Introduction to DreamFactory
  • 12:52 – NoSQL with DreamFactory
  • 16:10 – Use Case Discussion
  • 22:42 – Addition non-database integrations/transformations
  • 30:27 – Business logic in DreamFactory
  • 35:04 – Customer 360 Use Case
  • 43:03 – “Lightning Round” of technical topics: performance, indexing, logging/monitoring, profiling
If you’d like to learn more about the DreamFactory Platform while commuting or working out at the gym, download the 53 minute podcast now!

Posted on

Creating an Almost No Code Database-backed Web Interface with DreamFactory and Tabulator

No matter the size or industry, companies everywhere grapple with solving a deceptively simple problem: displaying database data on the web in a table. This problem seems to be universal in the sense that all teams have experienced the inconvenience of passing around Excel spreadsheets, and want to instead move this data to the web. Yet these same teams often lack the programming experience required to securely retrieve the data and present it within a paginated, filterable and often searchable web interface. Fortunately, using DreamFactory and off-the-shelf software such as Tabulator or DataTables, it’s possible to create a powerful database-backed web interface such as the one presented in the following screenshot:

Example dashboard interface

If you’d like to interact with a live example head over to http://tabulator.demo.dreamfactory.com/ and get your hands dirty! In this blog post I’ll show you how easy it is to create your own database-backed web interface using DreamFactory and Tabulator with almost no code required.

Creating the Database API

DreamFactory is an API generation and management platform used by thousands of organizations around the globe. You can download our open source version from DreamFactory.com or contact us to start an on premise or hosted trial of our commercial version. The open source edition includes support for several popular database, including MySQL and PostgreSQL, whereas the commercial edition additionally supports Microsoft SQL Server and Oracle. Once installed, you can generate a database-backed API in literally minutes, secured by (at minimum) an API and role-based access controls. If you’re not familiar with this process head over to DreamFactory Academy and watch the following video, “Creating Your First Database API with DreamFactory”:
You’ll also find dozens of other videos on our Youtube channel. Once the API created, you can query the database using a standardized API URL structure such as: 
https://example.com/api/v2/corporate/_table/employees
Of course, for security reasons you’ll need to additionally pass along an API key. This process is described in the aforementioned video.

Integrating the API with Tabulator

Tabulator is an open source JavaScript library used for creating interactive HTML tables. It supports data loading via a JavaScript array, JSON formatted data, or an AJAX data source. We’ll use the latter approach to load data from the DreamFactory-powered database API. To begin though, you’ll first need to install the library. If you’re looking for the easiest solution, we recommend cloning our dreamfactory-tabulator repository, located on GitHub. Alternatively a number of other installation solutions are supported (Bower, NPM, Yarn, etc); check out the Tabulator documentation for more information. Once installed, you only need to add a few lines of boilerplate code to load the database data into the Tabulator table. The following example contains just 35 lines of JavaScript code and will recreate a basic version of the interface presented in the earlier screenshot: 
<!DOCTYPE html>
<html>
    <head>
        <title>Tabulator example</title>
    </head>
    <body>
        <div id="example-table"></div>

        <script>

            var table = new Tabulator("#example-table", {
                    layout:"fitData",
                    height:"100%",
                    layout:"fitColumns",
                    pagination:"local",
                    paginationSize:20,
                    paginationSizeSelector:[20, 40, 60, 80],
                    placeholder:"No Data Set",
                    columns:[
                            {title:"Employee Number", field:"emp_no"},
                            {title:"First Name", field:"first_name"},
                            {title:"Last Name", field:"last_name"},
                            {title:"Birth Date", field:"birth_date"},
                            {title:"Hire Date", field:"hire_date"},
                    ],
            });

            $(document).ready(function(){
                    var ajaxConfig = {
                            dataType: 'json',
                            headers: {
                                    "X-DreamFactory-Api-Key": '123456qwerttasdf' 
                            },
                    };

                    table.setData(
                        'https://example.com/api/v2/corporate/_table/employees', 
                        {}, 
                        ajaxConfig
                    );
            });

        </script>
</body>
</html>
Let’s breakdown some of the key syntax found in this example:
  • The div identified by the example-table id will serve as the location where Tabulator will inject the table. When the Tabulator object is created, you’ll see this div ID (#example-table) is passed in as the first argument so Tabulator knows what div to use.
  • The columns defined in the columns array should match the fields returned within the DreamFactory JSON response. In this example, each record includes five fields: emp_no, first_name, last_name, birth_date, and hire_date.
  • The ajaxConfig object defines the X-DreamFactory-Api-Key header. You’ll assign the API key generated within DreamFactory to this header.
  • The setData method identifies the URL that will be contacted to retrieve the JSON data.
Check out a more involved example in our GitHub repository to learn how to add other features!

In Summary

If you’d like to learn more about DreamFactory, and how our platform can be used to quickly generate powerful web interfaces with almost no code required, contact us at DreamFactory.com!
Posted on

DreamFactory 3.0 Early Access Release Now Available

We are excited to announce availability of DreamFactory 3.0 early access release. This release is the culmination of several months of hard work, and is undoubtedly the most significant platform release in more than 2 years. This release is ready to be cloned via GitHub by way of the 3.0-beta branch. Docker users can spin up the 3.0 release made available via our Docker repository. So what’s new in this release? Let’s review the key features.

New User Interface

The most notable enhancement is a brand new interface. While the 2.X interface works just fine, it is shall we say rather utilitarian in nature. But there’s nothing wrong with adding a little aesthetic appeal, right? So while we’ve kept the user interface layout intact (for now), you’ll find that everything looks much shinier:

The new DreamFactory UI

By the way, the new UI release coincided with the launch of our completely revamped website (https://www.dreamfactory.com/). Be sure to check it out if you haven’t already!

Automated Linux “Genie” Installers

DreamFactory talks to a lot of data sources and third-party services, and for this reason developers have done so many amazing things with the platform. But installing and configuring all of the necessary drivers can be a pretty intimidating process. Our Docker environment goes a long way towards alleviating these issues however understandably not everybody is using Docker so we wanted to ease the installation process for other users. The result are two (soon to be 4) new installation scripts found in the installers directory. These installer scripts are used in conjunction with Debian and Ubuntu, and install all of the dependencies you need to run DreamFactory! If you’re running one of our commercial editions, the installers will additionally assist in the installation of the Microsoft SQL Server and Oracle drivers!

Python 3 Support

The DreamFactory Platform has long supported four scripting engines, including Node.js, PHP, Python, and V8JS. Scripting support is undoubtedly one of DreamFactory’s most popular features, not only because it’s used to add business logic to API endpoints, but also because you can create entirely new APIs using the Scripted Services connector. To put the popularity of this feature into perspective, we are aware of at least two customers who are running more than 400 scripted APIs within their DreamFactory environments! One of these customers uses Python exclusively, and requested Python 3 support not only because they wanted to take advantage of various libraries that have been ported to Python 3. Always happy to prioritize customer requests, we added Python 3 support to help them out and also because official Python 2.7 updates are slated to cease on January 1, 2020. For users running Python 2 scripts, have no fear! We added Python 3 support, and did not remove Python 2 support. When creating a new scripted service or adding logic to and endpoint, you’ll have the option of selecting between the two versions, as depicted in the following screenshot:

Python 3 Support

Updates to the GitHub, GitLab, and Bitbucket Connectors

There’s nothing particularly exciting to talk about here, other than we’ve additionally done quite a bit of maintenance work on the source control connectors, refactoring code and upgrading the Bitbucket integration to support the latest API release.

So What’s Next?

As we move towards a production release, we’ll be fixing bugs and adding a few more features to the platform:
  • Restricted Administration and API Auditing capabilities. We’re certainly burying the lede by casually mentioning this feature so late in the post. This is going to be a huge new addition to the platform, and will undoubtedly warrant a blog post unto itself.
  • Add CentOS and Fedora genie installers. These installers have been extensively tested by customers, and so in the coming weeks we’ll be adding it alongside the Debian and Ubuntu installers.
  • Upgrade installation scripts’ Oracle drivers to 18.5

We’d Love Your Feedback!

We’re very excited to share this release with the DreamFactory community, and would greatly appreciate your feedback! Please download, install, and try to break the software, sending comments, complaints, and suggestions to [email protected]!
Posted on

DreamFactory launches ‘Middle-Out Middleware’ with Gavin Belson

The name ‘Gavin Belson’ has earned many accolades over the years: serving as Chief Innovation Officer at Hooli, founding and co-leading Nucleus compression technology, and generally making the world a better place. Now, DreamFactory is proud to announce that Gavin Belson has joined its advisory board, bringing with him a plethora of experience to help guide and mentor the upcoming Middle-Out Middleware launch.




Arriving just in time for DreamFactory’s new product, Gavin has been central to understanding the problems facing today’s smart developers and why they need Middle-Out Middleware. “Middleware is a method of reducing the noise and software services between the end user device and the data sources that power applications. DreamFactory has already delivered a single layer of automated middleware for developers; but we are now excited to annouce a new product that changes all of that. Unlike regulare middleware, Middle-Out Middleware structures itself in such a way that devices and data sources are aligned for maximum ‘data stroke’ efficiency” Gavin stated at his latest press announcement, appearing via hologram at Hooli. He continued, “imagine a sheath cupping and pulling data from each data source, before moving to deliver the data to each device before again pulling the device data needs and moving back to the data sources. There are many efficiency losses. This is why Middle-Out Middleware changes the game. By using the same dedicated sheath to milk the data sources and device in a single smooth motion, amazing latency synergies are gained. Quite frankly, it makes everyone happy, faster.”

Gavin’s advocacy has seen a number of Fortune 500 companies queue up to prototype just what sort of synergies this could translate into. One company’s CTO, remaining under NDA, told us “[t]his is the big release we have all been waiting for“.

Interested in trialling Middle-Out Middleware yourself? Launch is officially on 1 April 2019, and for the low low price of $3m USD p.a., your company could be charging forward with a platform unleashing the benefits that come with a Weissman score of 5.2.

What does this really mean? Contact DreamFactory to find out more!



Cleared for release through Hooli Public Relations via Gavin Belson
Posted on

GET this! Seven Simple REST Client Examples for Retrieving API Data

You’ve installed DreamFactory and connected to your database, and executed a few test API calls in the API Docs interface. So what’s next? Fortunately DreamFactory has already taken care of autogenerating your API endpoints so you can skip right past that and proceed to creating a client which will talk to the DreamFactory-managed API.

In this article we will demonstrate seven simple REST client examples involving sending a GET request to an HTTP-based API using an API key for authentication. We will present examples demonstrating NodeJS, Python, Ruby, PHP, and Perl. Even if you’re not a DreamFactory user (you should check it out, our OSS version is available for free download) there’s still plenty to learn from this post! Continue reading “GET this! Seven Simple REST Client Examples for Retrieving API Data”